-- Logs begin at Mon 2022-06-27 17:38:29 UTC, end at Mon 2022-06-27 19:05:47 UTC. -- Jun 27 18:25:33.217897 dv-cinder-844601-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 27 18:25:33.226535 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 27 18:25:33.228551 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jun 27 18:25:33 2022] *** Jun 27 18:25:33.228551 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 27 18:25:33.228551 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 27 18:25:33.228551 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: nodename: dv-cinder-844601-6-iscsi Jun 27 18:25:33.228551 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: machine: x86_64 Jun 27 18:25:33.228551 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: clock source: unix Jun 27 18:25:33.228551 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: pcre jit disabled Jun 27 18:25:33.228551 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: detected number of CPU cores: 4 Jun 27 18:25:33.228551 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: current working directory: / Jun 27 18:25:33.228551 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: detected binary path: /usr/bin/uwsgi-core Jun 27 18:25:33.228551 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: your processes number limit is 47680 Jun 27 18:25:33.229128 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: your memory page size is 4096 bytes Jun 27 18:25:33.229128 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: detected max file descriptor number: 2048 Jun 27 18:25:33.229128 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: lock engine: pthread robust mutexes Jun 27 18:25:33.229128 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: thunder lock: enabled Jun 27 18:25:33.229128 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 27 18:25:33.229128 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 27 18:25:33.272809 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: Python main interpreter initialized at 0x5654db0349c0 Jun 27 18:25:33.272809 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: python threads support enabled Jun 27 18:25:33.272809 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: your server socket listen backlog is limited to 100 connections Jun 27 18:25:33.272809 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: your mercy for graceful operations on workers is 90 seconds Jun 27 18:25:33.273139 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: mapped 403077 bytes (393 KB) for 2 cores Jun 27 18:25:33.273213 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: *** Operational MODE: preforking *** Jun 27 18:25:33.273469 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: *** uWSGI is running in multiple interpreter mode *** Jun 27 18:25:33.273469 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: spawned uWSGI master process (pid: 80558) Jun 27 18:25:33.273662 dv-cinder-844601-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 27 18:25:33.274516 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: spawned uWSGI worker 1 (pid: 80569, cores: 1) Jun 27 18:25:33.274633 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: spawned uWSGI worker 2 (pid: 80570, cores: 1) Jun 27 18:25:33.274705 dv-cinder-844601-6-iscsi devstack@keystone.service[80558]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 27 18:25:38.749546 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 18:25:38.749546 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: from cryptography.utils import int_from_bytes Jun 27 18:25:38.750184 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 18:25:38.750184 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: from cryptography.utils import int_from_bytes Jun 27 18:25:38.750541 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 18:25:38.750541 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: from cryptography.utils import int_from_bytes Jun 27 18:25:38.750985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 18:25:38.750985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: from cryptography.utils import int_from_bytes Jun 27 18:25:38.799150 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 27 18:25:38.799339 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 27 18:25:38.799531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] command line args: [] {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 27 18:25:38.799798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 27 18:25:38.799875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 27 18:25:38.799982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ================================================================================ {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 27 18:25:38.800106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 27 18:25:38.800260 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] command line args: [] {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 27 18:25:38.800352 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] admin_token = **** {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.800465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] config_dir = [] {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.800550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 27 18:25:38.800654 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.800740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ================================================================================ {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 27 18:25:38.800822 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] config_source = [] {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.800893 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] admin_token = **** {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.801000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] debug = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.801140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] config_dir = [] {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.801224 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.801477 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.801584 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] config_source = [] {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.801677 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.801786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] debug = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.801892 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.801992 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.802121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.802216 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.802310 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.802464 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.802615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] list_limit = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.802785 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.802949 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] log_config_append = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.802949 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.803199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.803363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] log_dir = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.803527 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] list_limit = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.803645 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] log_file = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.803734 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] log_config_append = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.803856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.803936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.803936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] log_dir = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.804113 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.804241 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] log_file = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.804322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.804397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.804510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.804585 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.804702 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.804702 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.804887 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.804959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.805071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.805071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.805254 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.805334 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.805411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.805524 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.805603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.805678 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.805799 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.805878 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.805947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.806012 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.806080 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.806191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.806270 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.806347 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.806480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.806573 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.806727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.806834 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.807005 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.807082 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] publish_errors = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.807234 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.807466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.807544 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.807645 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.807722 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] publish_errors = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.807880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.807962 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.808111 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.808186 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.808271 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.808348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] standard_threads = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.808522 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.808600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.808760 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.808836 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.808948 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.809032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] standard_threads = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.809109 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] use_journal = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.809214 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.809291 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] use_json = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.809447 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] use_stderr = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.809523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.809622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.809705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] use_syslog = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.809801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] use_journal = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.809878 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.809978 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] use_json = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.810090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.810243 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] use_stderr = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.810338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] use_syslog = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.810435 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.810622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 18:25:38.811060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.811153 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.811235 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.811313 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.811383 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.811458 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.811528 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.811666 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.811753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.811911 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.811993 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.812156 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.812283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.812401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.812487 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.812636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.812755 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.812872 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.812955 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.813120 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.813204 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.813355 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.813431 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.813589 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.813665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.813836 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.813934 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.814072 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.814154 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.814315 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.814411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.814590 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.814675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.814830 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.814908 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.815064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.815141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.815315 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.815393 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.815560 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.815636 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.815797 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.815879 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.816050 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.816129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.816282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.816360 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.816513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.816589 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.816743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.816827 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.816978 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.817056 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.817219 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.817294 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.817450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.817528 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.817685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.817768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.817915 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.817992 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.818160 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.818238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.818428 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.818507 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.818645 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.818732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.818888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.818991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.819120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.819222 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.819355 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.819461 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.819588 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.819697 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.819826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.819940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.820100 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.820183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.820310 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.820385 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.820497 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.820570 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.820682 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.820763 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.820919 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.821002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.821145 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.821229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.821377 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.821465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.821604 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.821682 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.821837 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.821918 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.822071 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.822152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.822302 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.822406 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.822556 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.822637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.822792 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.823001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.823083 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.823207 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.823288 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.823361 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.823480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.823554 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.823687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.823774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.823916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.823994 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.824150 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.824233 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.824382 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.824460 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.824616 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.824693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.824843 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.824928 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.825088 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.825172 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.825320 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.825397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.825549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.825633 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.825782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.825872 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.826047 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.826566 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.826566 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.826711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.826783 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.826861 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.826933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.827002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.827125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.827201 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.827278 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.827424 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.827498 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.827642 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.827721 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.827881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.827965 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.828127 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.828205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.828364 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.828447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.828601 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.828679 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.828838 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.828926 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.829072 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.829149 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.829308 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.829386 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.829543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.829627 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.829777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.829859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.830010 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.830087 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.830247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.830332 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.830510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.830590 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.830754 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.830831 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.830993 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.831087 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.831234 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.831311 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.831467 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.831544 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.831702 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.831789 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.831938 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.832015 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.832167 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.832243 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.832396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.832482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.832626 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.832705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.832857 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.832933 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.833084 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.833169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.833320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.833403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.833553 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.833630 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.833795 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.833878 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.834030 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.834106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.834278 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.834354 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.834543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.834634 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.834785 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.834861 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.835017 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] auth.external = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.835097 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.835246 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.835333 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.835499 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] auth.external = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.835575 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.835733 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.835842 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.835978 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] auth.password = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.836062 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.836207 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] auth.token = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.836294 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.836447 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.836531 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] auth.password = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.836681 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.836759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] auth.token = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.836918 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.837003 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.837146 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.837229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.837392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.837469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.837624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.837713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.837857 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.837934 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.838091 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] credential.caching = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.838183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.838325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.838444 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.838588 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] credential.caching = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.838668 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.838831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.838909 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.839067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.839154 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.839310 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.839389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.839544 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.839621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.839788 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.839868 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.840026 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.840106 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.840264 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.840341 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.840514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.840594 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.840760 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.840837 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.841013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.841092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.841269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.841356 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.841510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.842106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.842368 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.842796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.842796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.842941 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] federation.caching = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.843012 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.843083 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.843154 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] federation.caching = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.843224 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.843302 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.843453 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.843539 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.843684 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.843770 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.843929 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.844017 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.844168 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.844248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.844423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.844502 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.844675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.844762 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.844928 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.845006 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.845166 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.845245 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.845412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.845501 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.845646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] identity.caching = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.845724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.845888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] identity.caching = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.845966 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.846240 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.846341 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.846436 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.846509 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.846618 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.846697 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.846856 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.846947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.847088 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.847166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.847325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.847406 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.847565 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.847654 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.847812 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.847890 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.848045 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.848124 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.848275 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.848361 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.848507 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.848585 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.848739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.848826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.848983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.849070 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.849226 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.849305 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.849471 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.849559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.849713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.849800 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.849959 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.850037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.850205 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.850283 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.850471 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.850577 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.850704 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.850781 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.850936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.851014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.851164 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.851250 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.851394 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.851475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.851630 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.851709 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.851865 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.851944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.852103 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.852180 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.852330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.852407 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.852574 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.852653 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.852816 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.852893 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.853050 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.853128 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.853284 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.853362 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.853525 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.853602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.853754 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.853833 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.854046 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.854132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.854205 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.854274 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.854459 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.854538 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.854698 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.854779 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.854934 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.855012 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.855168 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.855246 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.855403 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.855482 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.855642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.855729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.855885 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.856232 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.856382 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.856463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.856611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.856690 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.856839 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.856917 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.857088 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.857171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.857925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.858418 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.858509 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.858584 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.858670 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.858745 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.858816 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.858888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.user = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.858959 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.user = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.859030 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.859109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.859221 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.859301 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.859454 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.859534 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.859701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.859792 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.859948 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.860028 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.860193 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.860273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.860429 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.860516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.860656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.860735 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.860887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.860965 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.861119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.861267 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.861348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.861419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.861568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.861648 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.861811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.861897 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.862051 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.862130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.862294 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.862372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.862561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.862653 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.862805 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.862884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.863033 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.863112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.863277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.863400 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.863513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.863593 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.863749 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.863827 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.863981 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.864060 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.864215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.864295 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.864459 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.864537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.864704 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.864786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.864939 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.865017 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.865188 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.865266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.865413 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.865487 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.865656 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.865734 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.865893 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.865970 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.866123 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.866202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.866358 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.866455 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.866628 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.866707 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.866869 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.866948 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.867097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.867175 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.867332 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.867410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.867566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] resource.caching = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.867645 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] resource.caching = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.867811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.867891 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.868055 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.868133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.868294 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.868375 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.868539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.868623 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.868787 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.868865 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.869019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.869097 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.869262 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.869345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.869494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.869572 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.869732 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.869811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.869963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] role.caching = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.870040 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] role.caching = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.870207 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] role.driver = sql {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.870286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] role.driver = sql {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.870461 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.870543 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.870709 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.870787 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.870957 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.871037 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.871204 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.871283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.871451 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.871529 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.871695 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.871773 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.871937 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.872016 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.872182 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.872260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.872425 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.872509 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.872658 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.872738 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.873045 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.873421 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.873496 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.873568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.873568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.873708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.873780 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.873850 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.874016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.874105 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.874251 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.874331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.874522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.874603 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.874772 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.874863 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.875018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.875095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.875263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.875341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.875495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.875648 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.875748 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.875827 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.875997 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.876075 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.876235 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.876321 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.876463 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.876541 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.876695 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.876774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.876922 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.877008 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.877172 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.877251 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.877424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.877501 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.877667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.877752 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.877902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.877981 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.878135 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.878214 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.878366 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.878514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.878623 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] token.caching = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.878701 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] token.caching = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.878859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.878937 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.879104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.879190 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.879336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.879415 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.879589 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.879668 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.879834 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.879920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.880067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.880145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.880310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.880388 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.880548 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.880634 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.880793 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.880871 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.881026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.881104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.881263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.881348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.881493 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.881571 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.881735 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.881813 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.881977 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.882064 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.882206 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.882284 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.882465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.882546 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.882759 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.882850 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.883019 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] database.connection = **** {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.883106 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] database.connection = **** {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.883264 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.883354 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.883510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.883587 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.883746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.883831 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.883985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.884063 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.884219 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.884296 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.884455 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.884603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.884679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.884749 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.884918 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.884996 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.885163 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.885249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.885398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.885476 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.885639 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.885718 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.885872 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.885957 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.886116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.886195 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.886349 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.886475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.886612 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.886702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.886842 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.886921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.887080 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.887157 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.887309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.887394 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 18:25:38.887468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80569) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 27 18:25:38.887538 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 27 18:25:38.918624 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:38.918791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:38.918864 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:38.918960 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:38.919066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:38.919152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:38.928236 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:38.928333 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:38.956101 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:38.956326 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:38.957738 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:38.957974 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.099579 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.099819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.149504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.149644 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.149728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.149807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.214355 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.214608 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.214786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.214875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.215026 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.215112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.215252 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.215337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.215469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.215555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.215685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.215903 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.240944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.241174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.241400 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.241488 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.241652 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.241739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.241885 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.241970 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.242103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.242196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.242317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.242426 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.242576 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.242827 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.254092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.254754 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.266964 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.267189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.267275 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.267412 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.267514 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.267633 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.267716 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.267851 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.267933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.268065 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.268149 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.268279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.268362 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.268510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80569) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 18:25:39.285586 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.285781 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.285869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 18:25:39.285995 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 18:25:39.286114 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.286317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 18:25:39.286432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.286556 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 18:25:39.286637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.286828 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.286910 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.287086 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.287175 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.287339 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.287419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.287592 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.298712 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.298983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.299245 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.299328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.299517 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.299608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.299790 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.300072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.307862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.308440 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.311797 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.312071 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.312352 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.312434 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.312628 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.312710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.312906 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.313090 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.313168 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.313242 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.313412 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.313495 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.313675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.313766 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.313937 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.314020 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.314205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.314284 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.314486 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.314570 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.314762 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.315025 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.341114 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.341829 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.342821 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.343093 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.343482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.343759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.348370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.348761 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.349037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.349119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.349432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.349755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.356647 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.356924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.357888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.358166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.362092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.362375 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.362677 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.362759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.362954 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.363037 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.363227 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.363311 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.363499 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.363646 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.363777 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.363904 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.364040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.364167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.364295 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.364516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.383555 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.383777 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.383907 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.384050 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.388408 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.388878 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.392109 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.392343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.392506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.392737 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.397523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.397809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.401172 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.401469 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.404906 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.405197 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.405327 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.405464 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.405598 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.405725 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.405850 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.406000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.406139 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.406283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.406439 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.406604 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.425919 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.426198 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.426495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.426624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.426750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.426889 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.427019 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.427277 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.434682 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.435274 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.436546 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.436814 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.437164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.437439 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.440045 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.440324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.440603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.440733 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.440863 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.441003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.441138 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.441275 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.441470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.441755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.454657 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.454934 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.455210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.455337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.455486 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.455624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.455873 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.456152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.457017 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.457310 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.457614 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.457741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.457962 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.458260 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.476280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.476909 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.479576 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.480274 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.483318 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.484060 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.485048 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.485791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.486703 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.486983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.487275 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.487470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.487749 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.488043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.494613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.495382 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.496219 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.497013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.500040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.500318 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.500719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.500896 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.501029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.501169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.501481 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.501774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.510091 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.510370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.510662 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.510993 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.511276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.511558 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.518730 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.519005 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.519282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.519536 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80569) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 18:25:39.519620 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.519798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.519880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.520075 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.520156 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.520348 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.520431 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.520618 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.520701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.520952 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.521040 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.521229 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.521311 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.521495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.521577 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.521751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.521834 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.522023 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.522105 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.522278 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.522367 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.522584 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.522860 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.523119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80569) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 18:25:39.561429 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5654db0349c0 pid: 80570 (default app) Jun 27 18:25:39.562152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5654db0349c0 pid: 80569 (default app) Jun 27 18:25:39.564795 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05e062-2af0-475e-a06e-e451219e709e None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:39.565088 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05e062-2af0-475e-a06e-e451219e709e None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:39.565365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05e062-2af0-475e-a06e-e451219e709e None None] PATH_INFO: `/v3` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:39.569532 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Mon Jun 27 18:25:39 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:25:47.933812 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020271af-0b20-40e5-9ad7-b7c432e920f1 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:47.934421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020271af-0b20-40e5-9ad7-b7c432e920f1 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:47.934421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020271af-0b20-40e5-9ad7-b7c432e920f1 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:47.939235 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1/2] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:25:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:25:47.944607 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe236a7c-4137-4443-8d30-9d08f31e064b None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:47.945044 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe236a7c-4137-4443-8d30-9d08f31e064b None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:47.945156 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe236a7c-4137-4443-8d30-9d08f31e064b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:48.017000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: INFO dbcounter [None req-fe236a7c-4137-4443-8d30-9d08f31e064b None None] Registered counter for database keystone Jun 27 18:25:48.038042 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG oslo_db.sqlalchemy.engines [None req-fe236a7c-4137-4443-8d30-9d08f31e064b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80570) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 27 18:25:48.044193 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writer thread running {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 27 18:25:48.222889 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG passlib.handlers.bcrypt [None req-fe236a7c-4137-4443-8d30-9d08f31e064b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80570) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 27 18:25:48.223309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG passlib.handlers.bcrypt [None req-fe236a7c-4137-4443-8d30-9d08f31e064b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80570) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 27 18:25:48.278794 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-fe236a7c-4137-4443-8d30-9d08f31e064b None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:25:48.281438 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-fe236a7c-4137-4443-8d30-9d08f31e064b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:25:48.404594 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 2/3] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:25:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 27 18:25:48.534054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020271af-0b20-40e5-9ad7-b7c432e920f1 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:48.603710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: INFO dbcounter [None req-020271af-0b20-40e5-9ad7-b7c432e920f1 None None] Registered counter for database keystone Jun 27 18:25:48.624124 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG oslo_db.sqlalchemy.engines [None req-020271af-0b20-40e5-9ad7-b7c432e920f1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80569) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 27 18:25:48.630621 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writer thread running {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 27 18:25:48.749589 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020271af-0b20-40e5-9ad7-b7c432e920f1 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:48.753887 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f069bcd-3646-47f4-be98-f05babbc02ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:48.754577 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f069bcd-3646-47f4-be98-f05babbc02ae admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:48.754860 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f069bcd-3646-47f4-be98-f05babbc02ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:48.755131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f069bcd-3646-47f4-be98-f05babbc02ae admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:48.763232 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f069bcd-3646-47f4-be98-f05babbc02ae admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:25:48.815376 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0f069bcd-3646-47f4-be98-f05babbc02ae admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 27 18:25:48.815657 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f069bcd-3646-47f4-be98-f05babbc02ae admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:25:48.825062 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 2/4] 10.222.0.15 () {58 vars in 1324 bytes} [Mon Jun 27 18:25:48 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 18:25:48.831048 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe236a7c-4137-4443-8d30-9d08f31e064b None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:48.845355 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe236a7c-4137-4443-8d30-9d08f31e064b None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:48.849758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7499da2-a044-4935-bf15-26904a25da12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:48.850893 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7499da2-a044-4935-bf15-26904a25da12 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:48.851190 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7499da2-a044-4935-bf15-26904a25da12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:48.851466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7499da2-a044-4935-bf15-26904a25da12 admin admin] PATH_INFO: `/v3/projects/d9ffe361a6094e199bb35c633199d02f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:48.855537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7499da2-a044-4935-bf15-26904a25da12 admin admin] RBAC: Authorizing `identity:get_project(project_id=d9ffe361a6094e199bb35c633199d02f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:25:48.907131 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a7499da2-a044-4935-bf15-26904a25da12 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 27 18:25:48.907470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7499da2-a044-4935-bf15-26904a25da12 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:25:48.912015 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 3/5] 10.222.0.15 () {56 vars in 1207 bytes} [Mon Jun 27 18:25:48 2022] GET /identity/v3/projects/d9ffe361a6094e199bb35c633199d02f => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:25:50.908545 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb4664-9430-470b-ab08-f0d4fc8c0e1c None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:50.909094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb4664-9430-470b-ab08-f0d4fc8c0e1c None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:50.909094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb4664-9430-470b-ab08-f0d4fc8c0e1c None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:50.913159 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 3/6] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:25:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:25:50.918540 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7fe74-9b8c-4226-a5e1-c242cf31bd89 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:50.918921 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7fe74-9b8c-4226-a5e1-c242cf31bd89 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:50.919090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7fe74-9b8c-4226-a5e1-c242cf31bd89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:50.938173 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-3be7fe74-9b8c-4226-a5e1-c242cf31bd89 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:25:50.939710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-3be7fe74-9b8c-4226-a5e1-c242cf31bd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:25:50.947496 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 4/7] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:25:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 27 18:25:50.969797 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb4664-9430-470b-ab08-f0d4fc8c0e1c None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:50.981884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb4664-9430-470b-ab08-f0d4fc8c0e1c None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:50.986198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b52ca8-2fd2-46d6-b68c-dc7c4d9fed2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:50.986848 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b52ca8-2fd2-46d6-b68c-dc7c4d9fed2c admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:50.987131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b52ca8-2fd2-46d6-b68c-dc7c4d9fed2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:50.987408 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b52ca8-2fd2-46d6-b68c-dc7c4d9fed2c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:50.995010 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b52ca8-2fd2-46d6-b68c-dc7c4d9fed2c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:25:50.996814 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b52ca8-2fd2-46d6-b68c-dc7c4d9fed2c admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:25:51.006019 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 4/8] 10.222.0.15 () {58 vars in 1324 bytes} [Mon Jun 27 18:25:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 18:25:51.011504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7fe74-9b8c-4226-a5e1-c242cf31bd89 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:51.020926 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7fe74-9b8c-4226-a5e1-c242cf31bd89 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:51.025291 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd7a29-317a-44f6-be7e-41b2fd5d11e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:51.026134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd7a29-317a-44f6-be7e-41b2fd5d11e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:51.026453 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd7a29-317a-44f6-be7e-41b2fd5d11e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:51.026737 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd7a29-317a-44f6-be7e-41b2fd5d11e5 admin admin] PATH_INFO: `/v3/users/3d0ddf436e7443679ab397306fca9e90` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:51.028119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd7a29-317a-44f6-be7e-41b2fd5d11e5 admin admin] RBAC: Authorizing `identity:get_user(user_id=3d0ddf436e7443679ab397306fca9e90)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:25:51.029942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd7a29-317a-44f6-be7e-41b2fd5d11e5 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:25:51.031543 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 5/9] 10.222.0.15 () {56 vars in 1198 bytes} [Mon Jun 27 18:25:51 2022] GET /identity/v3/users/3d0ddf436e7443679ab397306fca9e90 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:25:53.096970 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab2607-736a-4d23-992a-f8a4877fdacb None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:53.097507 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab2607-736a-4d23-992a-f8a4877fdacb None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:53.097507 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab2607-736a-4d23-992a-f8a4877fdacb None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:53.098892 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 5/10] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:25:53.103819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516d167-7941-4b90-8316-4d99f531fc5b None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:53.104222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516d167-7941-4b90-8316-4d99f531fc5b None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:53.104386 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516d167-7941-4b90-8316-4d99f531fc5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:53.121069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-9516d167-7941-4b90-8316-4d99f531fc5b None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:25:53.122265 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-9516d167-7941-4b90-8316-4d99f531fc5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:25:53.153495 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fc367-6596-4f38-ab6e-90e0b5b07498 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:53.153781 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fc367-6596-4f38-ab6e-90e0b5b07498 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:53.154085 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fc367-6596-4f38-ab6e-90e0b5b07498 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:53.155693 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 6/11] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:25:53.160365 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270c166-5dda-4b2e-8b2e-b0c041733c9a None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:53.160650 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270c166-5dda-4b2e-8b2e-b0c041733c9a None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:53.160931 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270c166-5dda-4b2e-8b2e-b0c041733c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:53.199626 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG passlib.handlers.bcrypt [None req-2270c166-5dda-4b2e-8b2e-b0c041733c9a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80569) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 27 18:25:53.200059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG passlib.handlers.bcrypt [None req-2270c166-5dda-4b2e-8b2e-b0c041733c9a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80569) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 27 18:25:53.203604 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 6/12] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:25:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:25:53.226680 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516d167-7941-4b90-8316-4d99f531fc5b None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:53.246124 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-2270c166-5dda-4b2e-8b2e-b0c041733c9a None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:25:53.248359 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-2270c166-5dda-4b2e-8b2e-b0c041733c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:25:53.249975 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:25:53.328409 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516d167-7941-4b90-8316-4d99f531fc5b None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:53.348031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 7/13] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:25:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:25:53.362300 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd281653-ef09-40fe-837d-33ff7454890a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:53.363115 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd281653-ef09-40fe-837d-33ff7454890a None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:53.363399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd281653-ef09-40fe-837d-33ff7454890a None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:53.363736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd281653-ef09-40fe-837d-33ff7454890a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:53.370310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270c166-5dda-4b2e-8b2e-b0c041733c9a None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:53.376342 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd281653-ef09-40fe-837d-33ff7454890a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:25:53.387891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-cd281653-ef09-40fe-837d-33ff7454890a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:25:53.387891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:25:53.387891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:25:53.387891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:25:53.387891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:25:53.387891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:25:53.387891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:25:53.387891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:25:53.387891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:25:53.387891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:25:53.387891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:25:53.387891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:25:53.387891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:25:53.387891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:25:53.387891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:25:53.387891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:25:53.388925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:25:53.388925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:25:53.388925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:25:53.388925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:25:53.388925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:25:53.388925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:25:53.388925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:25:53.388925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:25:53.388925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:25:53.388925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:25:53.388925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:25:53.388925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:25:53.388925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:25:53.388925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:25:53.388925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:25:53.388925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:25:53.389729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:25:53.389729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:25:53.389729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:25:53.389729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:25:53.389729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:25:53.389729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:25:53.389729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:25:53.389729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:25:53.389729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:25:53.389729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:25:53.389729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:25:53.389729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:25:53.389729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 7/14] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:25:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:25:53.395298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd281653-ef09-40fe-837d-33ff7454890a None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:53.470331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270c166-5dda-4b2e-8b2e-b0c041733c9a None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:53.472629 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:25:53.474082 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:25:53.494222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd281653-ef09-40fe-837d-33ff7454890a None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:53.507332 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1134f57-82d2-48a5-8511-dcee2f8b2b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:53.508217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1134f57-82d2-48a5-8511-dcee2f8b2b57 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:53.508555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1134f57-82d2-48a5-8511-dcee2f8b2b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:53.508859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1134f57-82d2-48a5-8511-dcee2f8b2b57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:53.528319 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7184e-1821-4118-b2b4-3d19144d8be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:53.529551 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7184e-1821-4118-b2b4-3d19144d8be1 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:53.529845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7184e-1821-4118-b2b4-3d19144d8be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:53.530127 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7184e-1821-4118-b2b4-3d19144d8be1 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:53.530921 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7184e-1821-4118-b2b4-3d19144d8be1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:25:53.531264 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7184e-1821-4118-b2b4-3d19144d8be1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:25:53.533015 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7184e-1821-4118-b2b4-3d19144d8be1 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:25:53.544492 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 8/15] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:25:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:25:53.550456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7184e-1821-4118-b2b4-3d19144d8be1 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:53.579666 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1134f57-82d2-48a5-8511-dcee2f8b2b57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:25:53.581180 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1134f57-82d2-48a5-8511-dcee2f8b2b57 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:25:53.647587 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7184e-1821-4118-b2b4-3d19144d8be1 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:53.681529 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aace54-7619-422b-b8cf-6899831b2425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:53.681739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 8/16] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:25:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 18:25:53.682411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aace54-7619-422b-b8cf-6899831b2425 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:53.682697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aace54-7619-422b-b8cf-6899831b2425 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:53.682971 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aace54-7619-422b-b8cf-6899831b2425 None admin] PATH_INFO: `/v3/users/3d0ddf436e7443679ab397306fca9e90` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:53.684180 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aace54-7619-422b-b8cf-6899831b2425 None admin] RBAC: Authorizing `identity:get_user(user_id=3d0ddf436e7443679ab397306fca9e90)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:25:53.686179 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aace54-7619-422b-b8cf-6899831b2425 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:25:53.687989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 9/17] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:25:53 2022] GET /identity/v3/users/3d0ddf436e7443679ab397306fca9e90 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:25:53.688389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1134f57-82d2-48a5-8511-dcee2f8b2b57 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:53.691221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:25:53.693203 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aace54-7619-422b-b8cf-6899831b2425 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:53.708429 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:25:53.788356 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1134f57-82d2-48a5-8511-dcee2f8b2b57 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:53.797038 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aace54-7619-422b-b8cf-6899831b2425 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:53.821784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77a0e2-4c67-46ff-8ecc-b3ea113a6ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:53.822593 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77a0e2-4c67-46ff-8ecc-b3ea113a6ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:53.822873 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77a0e2-4c67-46ff-8ecc-b3ea113a6ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:53.823146 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77a0e2-4c67-46ff-8ecc-b3ea113a6ba7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:53.831197 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7e802-79e6-44fa-9161-86be2dcf73e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:53.831972 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7e802-79e6-44fa-9161-86be2dcf73e8 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:53.832256 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7e802-79e6-44fa-9161-86be2dcf73e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:53.832532 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7e802-79e6-44fa-9161-86be2dcf73e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:53.833709 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c7e802-79e6-44fa-9161-86be2dcf73e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:25:53.835196 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c7e802-79e6-44fa-9161-86be2dcf73e8 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:25:53.836801 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 10/18] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:25:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:25:53.843061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77a0e2-4c67-46ff-8ecc-b3ea113a6ba7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:25:53.843187 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7e802-79e6-44fa-9161-86be2dcf73e8 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:53.844530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77a0e2-4c67-46ff-8ecc-b3ea113a6ba7 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:25:53.866853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-3b77a0e2-4c67-46ff-8ecc-b3ea113a6ba7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:25:53.866853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:25:53.866853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:25:53.866853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:25:53.866853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:25:53.866853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:25:53.866853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:25:53.866853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:25:53.866853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:25:53.866853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:25:53.866853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:25:53.866853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:25:53.866853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:25:53.866853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:25:53.866853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:25:53.867725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:25:53.867725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:25:53.867725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:25:53.867725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:25:53.867725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:25:53.867725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:25:53.867725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:25:53.867725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:25:53.867725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:25:53.867725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:25:53.867725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:25:53.867725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:25:53.867725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:25:53.867725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:25:53.867725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:25:53.867725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:25:53.867725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:25:53.868514 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:25:53.868514 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:25:53.868514 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:25:53.868514 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:25:53.868514 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 9/19] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:25:53.874173 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77a0e2-4c67-46ff-8ecc-b3ea113a6ba7 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:53.923139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:25:53.931194 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:25:53.941769 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7e802-79e6-44fa-9161-86be2dcf73e8 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:53.975276 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035dbe51-f0fd-451d-8876-e3eb1875dedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:53.975473 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77a0e2-4c67-46ff-8ecc-b3ea113a6ba7 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:53.975928 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dbe51-f0fd-451d-8876-e3eb1875dedf None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:53.976232 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dbe51-f0fd-451d-8876-e3eb1875dedf None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:53.976553 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dbe51-f0fd-451d-8876-e3eb1875dedf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:53.977388 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035dbe51-f0fd-451d-8876-e3eb1875dedf None admin] RBAC: Adding query filter params (role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f, scope.domain.id=default, user.id=3d0ddf436e7443679ab397306fca9e90) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:25:53.977697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035dbe51-f0fd-451d-8876-e3eb1875dedf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:25:53.979169 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035dbe51-f0fd-451d-8876-e3eb1875dedf None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:25:53.986706 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 11/20] 10.222.0.15 () {56 vars in 1322 bytes} [Mon Jun 27 18:25:53 2022] GET /identity/v3/role_assignments?user.id=3d0ddf436e7443679ab397306fca9e90&scope.domain.id=default&role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:25:54.013851 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d290b6-9489-40f5-a1c9-192add964395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:54.014567 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d290b6-9489-40f5-a1c9-192add964395 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:54.014851 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d290b6-9489-40f5-a1c9-192add964395 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:54.015122 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d290b6-9489-40f5-a1c9-192add964395 None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:54.015863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d290b6-9489-40f5-a1c9-192add964395 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:25:54.016167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d290b6-9489-40f5-a1c9-192add964395 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:25:54.017591 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d290b6-9489-40f5-a1c9-192add964395 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:25:54.037817 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 10/21] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:25:56.309653 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c926e-bafb-4f73-a1ee-2db7c187cfdf None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:56.310194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c926e-bafb-4f73-a1ee-2db7c187cfdf None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:56.310498 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c926e-bafb-4f73-a1ee-2db7c187cfdf None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:56.311908 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 12/22] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:25:56.317707 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c2a66-26f1-4152-83bd-bb9921a3597f None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:56.318175 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c2a66-26f1-4152-83bd-bb9921a3597f None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:56.318252 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c2a66-26f1-4152-83bd-bb9921a3597f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:56.341255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-ea7c2a66-26f1-4152-83bd-bb9921a3597f None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:25:56.342705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-ea7c2a66-26f1-4152-83bd-bb9921a3597f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:25:56.440279 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 11/23] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:25:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:25:56.467427 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998503ec-d088-4767-9a76-3bfddaa3fcbb None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:56.467718 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998503ec-d088-4767-9a76-3bfddaa3fcbb None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:56.467988 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998503ec-d088-4767-9a76-3bfddaa3fcbb None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:56.469358 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 13/24] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:25:56.473618 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c2a66-26f1-4152-83bd-bb9921a3597f None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:56.486971 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2c313-fde6-447d-a1c0-9e429d97de7c None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:56.487259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2c313-fde6-447d-a1c0-9e429d97de7c None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:56.487542 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2c313-fde6-447d-a1c0-9e429d97de7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:56.509932 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-03a2c313-fde6-447d-a1c0-9e429d97de7c None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:25:56.511170 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-03a2c313-fde6-447d-a1c0-9e429d97de7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:25:56.591933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:25:56.672945 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 14/25] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:25:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 27 18:25:56.720496 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2c313-fde6-447d-a1c0-9e429d97de7c None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:56.721461 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c2a66-26f1-4152-83bd-bb9921a3597f None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:56.823131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ed8e6-599d-41ae-87c5-44c70cdfe1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:56.823929 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ed8e6-599d-41ae-87c5-44c70cdfe1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:56.824214 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ed8e6-599d-41ae-87c5-44c70cdfe1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:56.824490 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ed8e6-599d-41ae-87c5-44c70cdfe1d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:56.845663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:25:56.856875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ed8e6-599d-41ae-87c5-44c70cdfe1d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:25:56.866331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-3a8ed8e6-599d-41ae-87c5-44c70cdfe1d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:25:56.866331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:25:56.866331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:25:56.866331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:25:56.866331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:25:56.866331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:25:56.866331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:25:56.866331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:25:56.866331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:25:56.866331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:25:56.866331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:25:56.866331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:25:56.866331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:25:56.866331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:25:56.866331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:25:56.866331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:25:56.867199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:25:56.867199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:25:56.867199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:25:56.867199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:25:56.867199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:25:56.867199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:25:56.867199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:25:56.867199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:25:56.867199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:25:56.867199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:25:56.867199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:25:56.867199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:25:56.867199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:25:56.867199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:25:56.867199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:25:56.867199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:25:56.867972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:25:56.867972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:25:56.867972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:25:56.867972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:25:56.867972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:25:56.867972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:25:56.867972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:25:56.867972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:25:56.867972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:25:56.867972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:25:56.867972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:25:56.867972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:25:56.870354 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 12/26] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:25:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:25:56.871859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586346c1-7c29-42c9-8fb7-314558f4e1c4 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:56.872335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586346c1-7c29-42c9-8fb7-314558f4e1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:56.873359 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586346c1-7c29-42c9-8fb7-314558f4e1c4 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:56.875059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 13/27] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:25:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:25:56.878947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2c313-fde6-447d-a1c0-9e429d97de7c None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:56.881757 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b59c3-37e7-4d4a-b835-845d094fc41a None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:56.882256 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b59c3-37e7-4d4a-b835-845d094fc41a None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:56.882729 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b59c3-37e7-4d4a-b835-845d094fc41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:56.919444 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-8c6b59c3-37e7-4d4a-b835-845d094fc41a None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:25:56.920764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-8c6b59c3-37e7-4d4a-b835-845d094fc41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:25:56.924633 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453f39f-d1d2-4e3b-a59d-299f8498e1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:56.925425 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453f39f-d1d2-4e3b-a59d-299f8498e1ef None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:56.925706 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453f39f-d1d2-4e3b-a59d-299f8498e1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:56.925982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453f39f-d1d2-4e3b-a59d-299f8498e1ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:56.944597 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453f39f-d1d2-4e3b-a59d-299f8498e1ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:25:56.946089 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453f39f-d1d2-4e3b-a59d-299f8498e1ef None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:25:56.971329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-7453f39f-d1d2-4e3b-a59d-299f8498e1ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:25:56.971329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:25:56.971329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:25:56.971329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:25:56.971329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:25:56.971329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:25:56.971329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:25:56.971329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:25:56.971329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:25:56.971329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:25:56.971329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:25:56.971329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:25:56.971329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:25:56.971329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:25:56.971329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:25:56.972140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:25:56.972140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:25:56.972140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:25:56.972140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:25:56.972140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:25:56.972140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:25:56.972140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:25:56.972140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:25:56.972140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:25:56.972140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:25:56.972140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:25:56.972140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:25:56.972140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:25:56.972140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:25:56.972140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:25:56.972140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:25:56.972140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:25:56.972936 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:25:56.972936 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:25:56.972936 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:25:56.972936 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:25:56.972936 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 15/28] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:25:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:25:56.973615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453f39f-d1d2-4e3b-a59d-299f8498e1ef None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:57.054403 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 14/29] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:25:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:25:57.055263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b59c3-37e7-4d4a-b835-845d094fc41a None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:57.122257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:25:57.165613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453f39f-d1d2-4e3b-a59d-299f8498e1ef None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:57.203480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:25:57.215751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b42aba-5d16-40fe-9c51-cca207b33047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:57.217032 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b42aba-5d16-40fe-9c51-cca207b33047 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:57.217354 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b42aba-5d16-40fe-9c51-cca207b33047 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:57.217603 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b59c3-37e7-4d4a-b835-845d094fc41a None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:57.217714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b42aba-5d16-40fe-9c51-cca207b33047 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:57.218595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b42aba-5d16-40fe-9c51-cca207b33047 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:25:57.218921 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b42aba-5d16-40fe-9c51-cca207b33047 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:25:57.221487 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b42aba-5d16-40fe-9c51-cca207b33047 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:25:57.232984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 16/30] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:25:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:25:57.234738 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b42aba-5d16-40fe-9c51-cca207b33047 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:57.261113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb9043-812f-4c38-b63d-afacea19104a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:57.278569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb9043-812f-4c38-b63d-afacea19104a None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:57.278569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb9043-812f-4c38-b63d-afacea19104a None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:57.278569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb9043-812f-4c38-b63d-afacea19104a None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:57.278569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb9043-812f-4c38-b63d-afacea19104a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:25:57.278569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb9043-812f-4c38-b63d-afacea19104a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:25:57.278569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb9043-812f-4c38-b63d-afacea19104a None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:25:57.288049 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 15/31] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:25:57.288049 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955da80-745d-47f9-bada-224f78d83650 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:57.288273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955da80-745d-47f9-bada-224f78d83650 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:57.290287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955da80-745d-47f9-bada-224f78d83650 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:57.291617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 16/32] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:25:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:25:57.293934 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955da80-745d-47f9-bada-224f78d83650 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:57.377852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b42aba-5d16-40fe-9c51-cca207b33047 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:57.403213 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:25:57.410779 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955da80-745d-47f9-bada-224f78d83650 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:57.428510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac6eaa-e759-47d5-9ecf-7dc2f0729401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:57.429840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac6eaa-e759-47d5-9ecf-7dc2f0729401 None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:57.430331 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac6eaa-e759-47d5-9ecf-7dc2f0729401 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:57.430842 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac6eaa-e759-47d5-9ecf-7dc2f0729401 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:57.432219 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac6eaa-e759-47d5-9ecf-7dc2f0729401 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:25:57.434134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac6eaa-e759-47d5-9ecf-7dc2f0729401 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:25:57.448189 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480e677-25b1-48b6-be3b-458e53582a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:57.449091 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480e677-25b1-48b6-be3b-458e53582a40 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:57.449371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480e677-25b1-48b6-be3b-458e53582a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:57.449644 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480e677-25b1-48b6-be3b-458e53582a40 None admin] PATH_INFO: `/v3/users/3d0ddf436e7443679ab397306fca9e90` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:57.451419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480e677-25b1-48b6-be3b-458e53582a40 None admin] RBAC: Authorizing `identity:get_user(user_id=3d0ddf436e7443679ab397306fca9e90)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:25:57.452873 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480e677-25b1-48b6-be3b-458e53582a40 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:25:57.454587 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 17/33] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:25:57 2022] GET /identity/v3/users/3d0ddf436e7443679ab397306fca9e90 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:25:57.456059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480e677-25b1-48b6-be3b-458e53582a40 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:57.565311 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480e677-25b1-48b6-be3b-458e53582a40 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:57.601244 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a1803-53b9-491d-b027-db59b3fb6781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:57.601953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a1803-53b9-491d-b027-db59b3fb6781 None admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:57.602240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a1803-53b9-491d-b027-db59b3fb6781 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:57.602548 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a1803-53b9-491d-b027-db59b3fb6781 None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:57.603261 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a1803-53b9-491d-b027-db59b3fb6781 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:25:57.604738 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a1803-53b9-491d-b027-db59b3fb6781 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:25:57.675930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 17/34] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Jun 27 18:25:57 2022] POST /identity/v3/roles => generated 225 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:25:57.677503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35716f-5499-43af-9a10-474d118c53dc None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:57.677803 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35716f-5499-43af-9a10-474d118c53dc None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:57.678087 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35716f-5499-43af-9a10-474d118c53dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:57.702147 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-ea35716f-5499-43af-9a10-474d118c53dc None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:25:57.704026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ea35716f-5499-43af-9a10-474d118c53dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:25:57.796696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:25:57.830410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 18/35] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:25:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:25:57.830410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35716f-5499-43af-9a10-474d118c53dc None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:57.882738 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 18/36] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Jun 27 18:25:57 2022] POST /identity/v3/projects => generated 301 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:25:57.884273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3f88c-fdee-4eab-980f-47b92876a979 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:57.884571 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3f88c-fdee-4eab-980f-47b92876a979 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:57.884850 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3f88c-fdee-4eab-980f-47b92876a979 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:57.886131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 19/37] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:25:57.887388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d9300-8288-48c3-8715-860ae4651bbd None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:57.887735 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d9300-8288-48c3-8715-860ae4651bbd None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:57.888019 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d9300-8288-48c3-8715-860ae4651bbd None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:57.890073 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 20/38] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:25:57.891226 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d9300-8288-48c3-8715-860ae4651bbd None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:57.920268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:25:57.966803 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35716f-5499-43af-9a10-474d118c53dc None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:58.006053 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad1a19-8f2f-4e50-b9c4-befc10868a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:58.006894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad1a19-8f2f-4e50-b9c4-befc10868a3b None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:58.007183 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad1a19-8f2f-4e50-b9c4-befc10868a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:58.007462 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad1a19-8f2f-4e50-b9c4-befc10868a3b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:58.008730 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad1a19-8f2f-4e50-b9c4-befc10868a3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:25:58.010360 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad1a19-8f2f-4e50-b9c4-befc10868a3b None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:25:58.012414 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 19/39] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:25:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:25:58.014205 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef23b3-4f5b-4237-a997-6a1c82a13fee None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:58.014517 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef23b3-4f5b-4237-a997-6a1c82a13fee None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:58.014804 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef23b3-4f5b-4237-a997-6a1c82a13fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:58.026851 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d9300-8288-48c3-8715-860ae4651bbd None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:58.034749 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-74ef23b3-4f5b-4237-a997-6a1c82a13fee None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:25:58.035982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-74ef23b3-4f5b-4237-a997-6a1c82a13fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:25:58.073950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b2d60-f61f-43cd-a961-328d63a6266e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:58.074734 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b2d60-f61f-43cd-a961-328d63a6266e None admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:58.075020 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b2d60-f61f-43cd-a961-328d63a6266e None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:58.075293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b2d60-f61f-43cd-a961-328d63a6266e None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:58.076591 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b2d60-f61f-43cd-a961-328d63a6266e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:25:58.078101 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b2d60-f61f-43cd-a961-328d63a6266e None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:25:58.087577 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:25:58.112735 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 20/40] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:25:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:25:58.113997 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0fd783-9222-4845-bae4-54682eeeb993 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:58.114290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0fd783-9222-4845-bae4-54682eeeb993 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:58.114628 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0fd783-9222-4845-bae4-54682eeeb993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:58.130426 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-4c0fd783-9222-4845-bae4-54682eeeb993 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:25:58.131659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-4c0fd783-9222-4845-bae4-54682eeeb993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:25:58.151101 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 21/41] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Jun 27 18:25:57 2022] POST /identity/v3/roles => generated 219 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:25:58.151895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b2d60-f61f-43cd-a961-328d63a6266e None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:58.208672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 21/42] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:25:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:25:58.215435 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0fd783-9222-4845-bae4-54682eeeb993 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:58.246109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:25:58.271716 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b2d60-f61f-43cd-a961-328d63a6266e None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:58.316356 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6744f-5e2a-494c-994d-f860d80e4928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:58.317258 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6744f-5e2a-494c-994d-f860d80e4928 None admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:58.317546 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6744f-5e2a-494c-994d-f860d80e4928 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:58.317831 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6744f-5e2a-494c-994d-f860d80e4928 None admin] PATH_INFO: `/v3/domains/default/users/3d0ddf436e7443679ab397306fca9e90/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:58.321166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6744f-5e2a-494c-994d-f860d80e4928 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3d0ddf436e7443679ab397306fca9e90, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:25:58.322734 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6744f-5e2a-494c-994d-f860d80e4928 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:25:58.390043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 22/43] 10.222.0.15 () {58 vars in 1361 bytes} [Mon Jun 27 18:25:58 2022] PUT /identity/v3/domains/default/users/3d0ddf436e7443679ab397306fca9e90/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:25:58.390877 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6744f-5e2a-494c-994d-f860d80e4928 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:58.459560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0fd783-9222-4845-bae4-54682eeeb993 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:58.468397 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:25:58.503321 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1767f6-cf81-43b1-bd62-0cee3733ba81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:58.504488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1767f6-cf81-43b1-bd62-0cee3733ba81 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:58.505058 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1767f6-cf81-43b1-bd62-0cee3733ba81 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:58.505351 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1767f6-cf81-43b1-bd62-0cee3733ba81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:58.506655 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1767f6-cf81-43b1-bd62-0cee3733ba81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:25:58.506845 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6744f-5e2a-494c-994d-f860d80e4928 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:58.508170 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1767f6-cf81-43b1-bd62-0cee3733ba81 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:25:58.516410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 22/44] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:25:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:25:58.517369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1767f6-cf81-43b1-bd62-0cee3733ba81 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:25:58.554924 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081a8dc-6449-496b-8959-75f2ac69e179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:58.555602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081a8dc-6449-496b-8959-75f2ac69e179 None admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:58.555891 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081a8dc-6449-496b-8959-75f2ac69e179 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:58.556180 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081a8dc-6449-496b-8959-75f2ac69e179 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:58.556935 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081a8dc-6449-496b-8959-75f2ac69e179 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:25:58.558326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081a8dc-6449-496b-8959-75f2ac69e179 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:25:58.561813 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:25:58.625579 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1767f6-cf81-43b1-bd62-0cee3733ba81 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:25:58.660246 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c317fb3-476c-4481-ac6b-db8943468261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:25:58.660911 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c317fb3-476c-4481-ac6b-db8943468261 None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:25:58.661198 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c317fb3-476c-4481-ac6b-db8943468261 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:25:58.661477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c317fb3-476c-4481-ac6b-db8943468261 None admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:25:58.662253 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c317fb3-476c-4481-ac6b-db8943468261 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:25:58.663764 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c317fb3-476c-4481-ac6b-db8943468261 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:25:58.674489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 23/45] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Jun 27 18:25:58 2022] POST /identity/v3/roles => generated 223 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:25:58.758222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 23/46] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Jun 27 18:25:58 2022] POST /identity/v3/projects => generated 312 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:26:00.434079 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2323cb-2fc7-4072-b647-ff0a1b5c8593 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:00.434659 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2323cb-2fc7-4072-b647-ff0a1b5c8593 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:00.434659 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2323cb-2fc7-4072-b647-ff0a1b5c8593 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:00.436142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 24/47] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:00.441575 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06b637-7e2c-454d-82f5-6924eeebdd13 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:00.441986 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06b637-7e2c-454d-82f5-6924eeebdd13 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:00.442257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06b637-7e2c-454d-82f5-6924eeebdd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:00.458987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-1c06b637-7e2c-454d-82f5-6924eeebdd13 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:00.460173 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-1c06b637-7e2c-454d-82f5-6924eeebdd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:00.480724 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:00.527386 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 24/48] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:00.551599 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2323cb-2fc7-4072-b647-ff0a1b5c8593 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:00.650470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2323cb-2fc7-4072-b647-ff0a1b5c8593 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:00.685707 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ba433-500e-47fe-8c54-bc7540781733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:00.686474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ba433-500e-47fe-8c54-bc7540781733 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:00.686762 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ba433-500e-47fe-8c54-bc7540781733 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:00.687037 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ba433-500e-47fe-8c54-bc7540781733 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:00.696955 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ba433-500e-47fe-8c54-bc7540781733 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:00.705098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-3e6ba433-500e-47fe-8c54-bc7540781733 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:26:00.705098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:00.705098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:00.705098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:00.705098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:00.705098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:00.705098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:00.705098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:00.705098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:00.705098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:00.705098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:00.705098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:00.705098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:00.705098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:00.705098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:00.705098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:00.705936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:00.705936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:00.705936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:00.705936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:00.705936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:00.705936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:00.705936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:00.705936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:00.705936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:00.705936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:00.705936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:00.705936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:00.705936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:00.705936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:00.705936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:00.705936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:00.706723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:00.706723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:00.706723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:00.706723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:00.706723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:00.706723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:00.706723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:00.706723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:00.706723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:00.706723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:00.706723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:26:00.706723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:26:00.706723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 25/49] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:26:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:00.712158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06b637-7e2c-454d-82f5-6924eeebdd13 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:00.775999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9b4c2-4d59-483d-a1ff-1ad25ae1b1b5 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:00.776281 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9b4c2-4d59-483d-a1ff-1ad25ae1b1b5 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:00.776559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9b4c2-4d59-483d-a1ff-1ad25ae1b1b5 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:00.777919 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 26/50] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:26:00.782625 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5aba9-1ae5-40db-a95e-08e49667db43 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:00.782915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5aba9-1ae5-40db-a95e-08e49667db43 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:00.783197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5aba9-1ae5-40db-a95e-08e49667db43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:00.800061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-1be5aba9-1ae5-40db-a95e-08e49667db43 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:00.801567 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-1be5aba9-1ae5-40db-a95e-08e49667db43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:00.818776 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06b637-7e2c-454d-82f5-6924eeebdd13 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:00.850944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:00.852807 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae891de-5945-48da-90f8-19c814ec3124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:00.853473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae891de-5945-48da-90f8-19c814ec3124 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:00.853761 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae891de-5945-48da-90f8-19c814ec3124 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:00.854041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae891de-5945-48da-90f8-19c814ec3124 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:00.854833 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae891de-5945-48da-90f8-19c814ec3124 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:00.855145 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae891de-5945-48da-90f8-19c814ec3124 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:00.856799 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae891de-5945-48da-90f8-19c814ec3124 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:00.866067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 25/51] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:00.867959 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 27/52] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:26:00.871565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae891de-5945-48da-90f8-19c814ec3124 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:00.890829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:00.891297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5aba9-1ae5-40db-a95e-08e49667db43 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:00.971277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae891de-5945-48da-90f8-19c814ec3124 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:00.991455 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5aba9-1ae5-40db-a95e-08e49667db43 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:01.004831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625d5bd-25ca-461a-84f3-78a2dc7e65f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:01.005698 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625d5bd-25ca-461a-84f3-78a2dc7e65f3 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:01.005978 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625d5bd-25ca-461a-84f3-78a2dc7e65f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:01.006255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625d5bd-25ca-461a-84f3-78a2dc7e65f3 None admin] PATH_INFO: `/v3/users/3d0ddf436e7443679ab397306fca9e90` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:01.007508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625d5bd-25ca-461a-84f3-78a2dc7e65f3 None admin] RBAC: Authorizing `identity:get_user(user_id=3d0ddf436e7443679ab397306fca9e90)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:01.008965 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625d5bd-25ca-461a-84f3-78a2dc7e65f3 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:01.010826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 26/53] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:00 2022] GET /identity/v3/users/3d0ddf436e7443679ab397306fca9e90 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:01.016631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625d5bd-25ca-461a-84f3-78a2dc7e65f3 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:01.028055 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19458b0e-cd33-4169-b682-d88d3a5ce524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:01.028952 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19458b0e-cd33-4169-b682-d88d3a5ce524 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:01.029746 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19458b0e-cd33-4169-b682-d88d3a5ce524 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:01.030024 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19458b0e-cd33-4169-b682-d88d3a5ce524 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:01.031214 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19458b0e-cd33-4169-b682-d88d3a5ce524 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:01.032672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19458b0e-cd33-4169-b682-d88d3a5ce524 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:01.034274 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 28/54] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:26:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:01.039623 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19458b0e-cd33-4169-b682-d88d3a5ce524 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:01.094654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:01.102167 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:01.116963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625d5bd-25ca-461a-84f3-78a2dc7e65f3 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:01.137618 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19458b0e-cd33-4169-b682-d88d3a5ce524 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:01.150052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0206aa-d6fc-4eab-8627-eb53bab306e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:01.150860 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0206aa-d6fc-4eab-8627-eb53bab306e9 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:01.151140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0206aa-d6fc-4eab-8627-eb53bab306e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:01.151413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0206aa-d6fc-4eab-8627-eb53bab306e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:01.152627 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0206aa-d6fc-4eab-8627-eb53bab306e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:01.154828 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0206aa-d6fc-4eab-8627-eb53bab306e9 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:01.156441 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 27/55] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:26:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:01.162167 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0206aa-d6fc-4eab-8627-eb53bab306e9 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:01.173360 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19efa89d-de24-402a-8c4b-994e32dc8571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:01.174013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efa89d-de24-402a-8c4b-994e32dc8571 None admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:01.174303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efa89d-de24-402a-8c4b-994e32dc8571 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:01.174606 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efa89d-de24-402a-8c4b-994e32dc8571 None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:01.175405 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19efa89d-de24-402a-8c4b-994e32dc8571 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:01.176854 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19efa89d-de24-402a-8c4b-994e32dc8571 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:01.274847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0206aa-d6fc-4eab-8627-eb53bab306e9 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:01.309948 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5725a0f-5a80-42bb-8359-a41e85dd2fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:01.310817 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5725a0f-5a80-42bb-8359-a41e85dd2fbd None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:01.311117 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5725a0f-5a80-42bb-8359-a41e85dd2fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:01.311386 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5725a0f-5a80-42bb-8359-a41e85dd2fbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:01.312216 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5725a0f-5a80-42bb-8359-a41e85dd2fbd None admin] RBAC: Adding query filter params (role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f, scope.domain.id=default, user.id=3d0ddf436e7443679ab397306fca9e90) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:01.312528 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5725a0f-5a80-42bb-8359-a41e85dd2fbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:01.313972 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5725a0f-5a80-42bb-8359-a41e85dd2fbd None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:01.319669 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 28/56] 10.222.0.15 () {56 vars in 1322 bytes} [Mon Jun 27 18:26:01 2022] GET /identity/v3/role_assignments?user.id=3d0ddf436e7443679ab397306fca9e90&scope.domain.id=default&role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 530 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:01.465800 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 29/57] 10.222.0.15 () {60 vars in 1140 bytes} [Mon Jun 27 18:26:01 2022] POST /identity/v3/projects => generated 298 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:26:03.505053 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3c1c0-49b4-4657-8c6c-355a87b63797 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:03.505573 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3c1c0-49b4-4657-8c6c-355a87b63797 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:03.505657 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3c1c0-49b4-4657-8c6c-355a87b63797 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:03.507008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 29/58] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:03.512200 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649bb6c-8d99-4ef7-988a-42527f3c1adf None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:03.512594 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649bb6c-8d99-4ef7-988a-42527f3c1adf None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:03.512755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649bb6c-8d99-4ef7-988a-42527f3c1adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:03.530023 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-4649bb6c-8d99-4ef7-988a-42527f3c1adf None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:03.531258 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-4649bb6c-8d99-4ef7-988a-42527f3c1adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:03.594713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 30/59] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:03.618861 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3c1c0-49b4-4657-8c6c-355a87b63797 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:03.637710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:03.716873 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3c1c0-49b4-4657-8c6c-355a87b63797 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:03.749972 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b6071-3135-4aa8-a798-c9c7da2249d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:03.750803 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b6071-3135-4aa8-a798-c9c7da2249d7 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:03.751091 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b6071-3135-4aa8-a798-c9c7da2249d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:03.751370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b6071-3135-4aa8-a798-c9c7da2249d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:03.752563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7b6071-3135-4aa8-a798-c9c7da2249d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:03.754026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7b6071-3135-4aa8-a798-c9c7da2249d7 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:03.755746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 30/60] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:26:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:03.761056 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649bb6c-8d99-4ef7-988a-42527f3c1adf None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:03.795076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:03.858920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649bb6c-8d99-4ef7-988a-42527f3c1adf None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:03.891450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92517723-7438-4396-9c5e-681da4400e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:03.892101 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92517723-7438-4396-9c5e-681da4400e79 None admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:03.892385 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92517723-7438-4396-9c5e-681da4400e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:03.892663 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92517723-7438-4396-9c5e-681da4400e79 None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:03.893368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92517723-7438-4396-9c5e-681da4400e79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:03.894846 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92517723-7438-4396-9c5e-681da4400e79 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:03.989260 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 31/61] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Jun 27 18:26:03 2022] POST /identity/v3/users => generated 275 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:26:07.206197 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc195856-03d1-46e7-ae1a-f063951d7398 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:07.207218 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc195856-03d1-46e7-ae1a-f063951d7398 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:07.207731 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc195856-03d1-46e7-ae1a-f063951d7398 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:07.209621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 31/62] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:07.214573 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b1edf-9031-4a91-86ac-1358f15f54d4 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:07.215381 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b1edf-9031-4a91-86ac-1358f15f54d4 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:07.215885 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b1edf-9031-4a91-86ac-1358f15f54d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:07.243441 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-1d3b1edf-9031-4a91-86ac-1358f15f54d4 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:07.245014 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-1d3b1edf-9031-4a91-86ac-1358f15f54d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:07.248352 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869202cf-861c-482e-95f3-b547bb733877 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:07.248861 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869202cf-861c-482e-95f3-b547bb733877 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:07.249356 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869202cf-861c-482e-95f3-b547bb733877 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:07.250970 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 32/63] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:07.256098 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed968b-4bb0-4940-9714-0e5320b06762 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:07.256627 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed968b-4bb0-4940-9714-0e5320b06762 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:07.257107 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed968b-4bb0-4940-9714-0e5320b06762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:07.281880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-b5ed968b-4bb0-4940-9714-0e5320b06762 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:07.283611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-b5ed968b-4bb0-4940-9714-0e5320b06762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:07.383050 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 32/64] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:07.406973 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b1edf-9031-4a91-86ac-1358f15f54d4 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:07.416072 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 33/65] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:26:07.423444 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2afc7-cbfd-46c0-b4da-35a172b2b694 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:07.423738 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2afc7-cbfd-46c0-b4da-35a172b2b694 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:07.424022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2afc7-cbfd-46c0-b4da-35a172b2b694 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:07.425649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 34/66] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:07.430792 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f764a9-35b6-403c-93ce-2234958ba9a8 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:07.430792 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f764a9-35b6-403c-93ce-2234958ba9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:07.431102 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f764a9-35b6-403c-93ce-2234958ba9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:07.434737 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:07.456441 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-c8f764a9-35b6-403c-93ce-2234958ba9a8 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:07.458002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c8f764a9-35b6-403c-93ce-2234958ba9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:07.472234 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:07.530054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b1edf-9031-4a91-86ac-1358f15f54d4 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:07.558411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 35/67] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:07.558411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f764a9-35b6-403c-93ce-2234958ba9a8 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:07.590300 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef9eb2-cd60-4c53-9632-e92174f9ae95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:07.591142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef9eb2-cd60-4c53-9632-e92174f9ae95 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:07.591534 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef9eb2-cd60-4c53-9632-e92174f9ae95 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:07.591987 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef9eb2-cd60-4c53-9632-e92174f9ae95 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:07.603173 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef9eb2-cd60-4c53-9632-e92174f9ae95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:07.611880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-b1ef9eb2-cd60-4c53-9632-e92174f9ae95 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:07.611880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:07.611880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:07.611880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:07.611880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:07.611880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:07.611880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:07.611880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:07.611880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:07.611880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:07.611880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:07.611880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:07.611880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:07.611880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:07.611880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:07.611880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:07.612739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:07.612739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:07.612739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:07.612739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:07.612739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:07.612739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:07.612739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:07.612739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:07.612739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:07.612739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:07.612739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:07.612739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:07.612739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:07.612739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:07.612739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:07.612739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:07.613500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:07.613500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:07.613500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:07.613500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:07.613500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:07.613500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:07.613500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:07.613500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:07.613500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:07.613500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:07.613500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:07.613500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:26:07.613500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 33/68] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:07.614767 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef9eb2-cd60-4c53-9632-e92174f9ae95 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:07.664746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f764a9-35b6-403c-93ce-2234958ba9a8 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:07.699673 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8878ee81-4635-4505-abc5-f76c18791b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:07.700475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878ee81-4635-4505-abc5-f76c18791b6e None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:07.701308 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878ee81-4635-4505-abc5-f76c18791b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:07.701702 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878ee81-4635-4505-abc5-f76c18791b6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:07.711144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:07.713741 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8878ee81-4635-4505-abc5-f76c18791b6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:07.721697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-8878ee81-4635-4505-abc5-f76c18791b6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:26:07.721697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:07.721697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:07.721697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:07.721697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:07.721697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:07.721697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:07.721697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:07.721697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:07.721697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:07.721697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:07.721697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:07.721697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:07.721697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:07.721697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:07.721697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:07.727043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:07.727043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:07.727043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:07.727043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:07.727043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:07.727043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:07.727043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:07.727043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:07.727043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:07.727043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:07.727043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:07.727043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:07.727043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:07.727043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:07.727043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:07.727043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:07.727883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:07.727883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:07.727883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:07.727883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:07.727883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:07.727883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:07.727883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:07.727883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:07.727883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:07.727883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:07.727883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:26:07.727883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:07.727883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 36/69] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:26:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:07.727883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63374e-a304-443b-a774-a2818cd62636 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:07.728572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63374e-a304-443b-a774-a2818cd62636 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:07.728572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63374e-a304-443b-a774-a2818cd62636 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:07.728572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 37/70] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:07.729820 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63374e-a304-443b-a774-a2818cd62636 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:07.735208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef9eb2-cd60-4c53-9632-e92174f9ae95 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:07.745671 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:07.773738 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac11f52-33ef-4b1a-a4c9-fcb3a68b91e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:07.774576 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac11f52-33ef-4b1a-a4c9-fcb3a68b91e9 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:07.774858 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac11f52-33ef-4b1a-a4c9-fcb3a68b91e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:07.775128 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac11f52-33ef-4b1a-a4c9-fcb3a68b91e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:07.785746 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac11f52-33ef-4b1a-a4c9-fcb3a68b91e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:07.793986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-5ac11f52-33ef-4b1a-a4c9-fcb3a68b91e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:07.793986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:07.793986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:07.793986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:07.793986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:07.793986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:07.793986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:07.793986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:07.793986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:07.793986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:07.793986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:07.793986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:07.793986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:07.793986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:07.793986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:07.793986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:07.795372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:07.795372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:07.795372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:07.795372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:07.795372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:07.795372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:07.795372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:07.795372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:07.795372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:07.795372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:07.795372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:07.795372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:07.795372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:07.795372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:07.795372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:07.795372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:07.801209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:07.801209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:07.801209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:07.801209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:07.801209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:07.801209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:07.801209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:07.801209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:07.801209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:07.801209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:07.801209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:07.801209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:26:07.801209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 34/71] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Jun 27 18:26:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:07.801209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac11f52-33ef-4b1a-a4c9-fcb3a68b91e9 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:07.849385 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63374e-a304-443b-a774-a2818cd62636 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:07.884481 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908bedb-85ce-4560-83fa-ce3caca9d137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:07.885141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908bedb-85ce-4560-83fa-ce3caca9d137 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:07.885432 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908bedb-85ce-4560-83fa-ce3caca9d137 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:07.885713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908bedb-85ce-4560-83fa-ce3caca9d137 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:07.886505 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908bedb-85ce-4560-83fa-ce3caca9d137 None admin] RBAC: Adding query filter params (name=member) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:07.886817 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908bedb-85ce-4560-83fa-ce3caca9d137 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:07.888261 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908bedb-85ce-4560-83fa-ce3caca9d137 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:07.896589 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 38/72] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:07.898071 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50681fd2-01ba-4b55-afe5-94c612b187f0 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:07.898366 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50681fd2-01ba-4b55-afe5-94c612b187f0 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:07.898698 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50681fd2-01ba-4b55-afe5-94c612b187f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:07.906509 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac11f52-33ef-4b1a-a4c9-fcb3a68b91e9 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:07.914437 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-50681fd2-01ba-4b55-afe5-94c612b187f0 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:07.915652 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-50681fd2-01ba-4b55-afe5-94c612b187f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:07.940632 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1edc4ee-dc32-4af0-a4fd-5481982e8d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:07.941298 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edc4ee-dc32-4af0-a4fd-5481982e8d3d None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:07.941582 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edc4ee-dc32-4af0-a4fd-5481982e8d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:07.941869 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1edc4ee-dc32-4af0-a4fd-5481982e8d3d None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:07.942643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1edc4ee-dc32-4af0-a4fd-5481982e8d3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:07.942954 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1edc4ee-dc32-4af0-a4fd-5481982e8d3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:07.944337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1edc4ee-dc32-4af0-a4fd-5481982e8d3d None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:07.956077 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 35/73] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:07.956898 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1edc4ee-dc32-4af0-a4fd-5481982e8d3d None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:07.964271 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:07.980263 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:07.996385 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 39/74] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:07.996385 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae4611-bd92-457b-abc0-3a6bb39b449a None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:07.996385 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae4611-bd92-457b-abc0-3a6bb39b449a None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:07.996385 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae4611-bd92-457b-abc0-3a6bb39b449a None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:07.996385 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 40/75] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:07.996385 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae4611-bd92-457b-abc0-3a6bb39b449a None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:08.063461 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1edc4ee-dc32-4af0-a4fd-5481982e8d3d None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:08.097073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae4611-bd92-457b-abc0-3a6bb39b449a None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:08.097236 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcedce8-a014-4d2f-a8e9-a6198b0bbfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:08.097730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcedce8-a014-4d2f-a8e9-a6198b0bbfe7 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:08.098016 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcedce8-a014-4d2f-a8e9-a6198b0bbfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:08.098293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcedce8-a014-4d2f-a8e9-a6198b0bbfe7 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:08.099072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcedce8-a014-4d2f-a8e9-a6198b0bbfe7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:08.099384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcedce8-a014-4d2f-a8e9-a6198b0bbfe7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:08.100756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcedce8-a014-4d2f-a8e9-a6198b0bbfe7 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:08.109596 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 36/76] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:26:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:08.110183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcedce8-a014-4d2f-a8e9-a6198b0bbfe7 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:08.134798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b6c50-a9b2-4320-8a99-f38c0584c6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:08.135664 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b6c50-a9b2-4320-8a99-f38c0584c6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:08.135948 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b6c50-a9b2-4320-8a99-f38c0584c6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:08.136227 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b6c50-a9b2-4320-8a99-f38c0584c6c0 None admin] PATH_INFO: `/v3/users/69a2de33463a462ca1305bf7bea04a37` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:08.149147 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b6c50-a9b2-4320-8a99-f38c0584c6c0 None admin] RBAC: Authorizing `identity:get_user(user_id=69a2de33463a462ca1305bf7bea04a37)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:08.150621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b6c50-a9b2-4320-8a99-f38c0584c6c0 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:08.152282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 41/77] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:07 2022] GET /identity/v3/users/69a2de33463a462ca1305bf7bea04a37 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:08.153587 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c7f83-5034-4711-8ee1-9832aec27238 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:08.154416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c7f83-5034-4711-8ee1-9832aec27238 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:08.154670 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c7f83-5034-4711-8ee1-9832aec27238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:08.171038 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-460c7f83-5034-4711-8ee1-9832aec27238 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:08.172220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-460c7f83-5034-4711-8ee1-9832aec27238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:08.174625 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:08.208686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcedce8-a014-4d2f-a8e9-a6198b0bbfe7 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:08.221829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:08.238746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 42/78] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:08.239613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c7f83-5034-4711-8ee1-9832aec27238 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:08.243687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959479b5-d8af-4a51-944f-5ae12dca76ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:08.245053 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959479b5-d8af-4a51-944f-5ae12dca76ae None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:08.245485 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959479b5-d8af-4a51-944f-5ae12dca76ae None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:08.245906 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959479b5-d8af-4a51-944f-5ae12dca76ae None admin] PATH_INFO: `/v3/users/3d0ddf436e7443679ab397306fca9e90` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:08.247313 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959479b5-d8af-4a51-944f-5ae12dca76ae None admin] RBAC: Authorizing `identity:get_user(user_id=3d0ddf436e7443679ab397306fca9e90)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:08.249495 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959479b5-d8af-4a51-944f-5ae12dca76ae None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:08.251491 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 37/79] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:08 2022] GET /identity/v3/users/3d0ddf436e7443679ab397306fca9e90 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:08.252404 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959479b5-d8af-4a51-944f-5ae12dca76ae None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:08.342828 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c7f83-5034-4711-8ee1-9832aec27238 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:08.352028 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959479b5-d8af-4a51-944f-5ae12dca76ae None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:08.376955 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b8f9d-a6d3-4041-ab28-9b452e3a475c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:08.377738 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b8f9d-a6d3-4041-ab28-9b452e3a475c None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:08.378022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b8f9d-a6d3-4041-ab28-9b452e3a475c None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:08.378305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b8f9d-a6d3-4041-ab28-9b452e3a475c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:08.385815 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617c3da-8021-44fc-b50d-4567c11b422c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:08.386713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617c3da-8021-44fc-b50d-4567c11b422c None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:08.387001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617c3da-8021-44fc-b50d-4567c11b422c None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:08.387280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617c3da-8021-44fc-b50d-4567c11b422c None admin] PATH_INFO: `/v3/users/69a2de33463a462ca1305bf7bea04a37` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:08.388597 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b8f9d-a6d3-4041-ab28-9b452e3a475c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:08.388993 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617c3da-8021-44fc-b50d-4567c11b422c None admin] RBAC: Authorizing `identity:get_user(user_id=69a2de33463a462ca1305bf7bea04a37)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:08.390473 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617c3da-8021-44fc-b50d-4567c11b422c None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:08.392104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 38/80] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:08 2022] GET /identity/v3/users/69a2de33463a462ca1305bf7bea04a37 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:08.393966 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617c3da-8021-44fc-b50d-4567c11b422c None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:08.396154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-907b8f9d-a6d3-4041-ab28-9b452e3a475c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:08.396154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:08.396154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:08.396154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:08.396154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:08.396154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:08.396154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:08.396154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:08.396154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:08.396154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:08.396154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:08.396154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:08.396154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:08.396154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:08.396154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:08.396154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:08.396961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:08.396961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:08.396961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:08.396961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:08.396961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:08.396961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:08.396961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:08.396961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:08.396961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:08.396961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:08.396961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:08.396961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:08.396961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:08.396961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:08.396961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:08.396961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:08.397711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:08.397711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:08.397711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:08.397711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:08.397711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:08.397711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:08.397711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:08.397711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:08.397711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:08.397711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:08.397711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:08.397711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:08.398295 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:08.398367 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 43/81] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:08.399605 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b8f9d-a6d3-4041-ab28-9b452e3a475c None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:08.446460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:08.493276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617c3da-8021-44fc-b50d-4567c11b422c None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:08.500459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b8f9d-a6d3-4041-ab28-9b452e3a475c None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:08.532572 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb811cb5-9200-407d-8944-247aa4dd2ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:08.533218 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb811cb5-9200-407d-8944-247aa4dd2ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:08.533508 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb811cb5-9200-407d-8944-247aa4dd2ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:08.533784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb811cb5-9200-407d-8944-247aa4dd2ef8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:08.534327 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4b0f0-a61a-44c4-a7c1-eb0d131ba115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:08.535014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4b0f0-a61a-44c4-a7c1-eb0d131ba115 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:08.535305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4b0f0-a61a-44c4-a7c1-eb0d131ba115 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:08.535586 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4b0f0-a61a-44c4-a7c1-eb0d131ba115 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:08.603172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb811cb5-9200-407d-8944-247aa4dd2ef8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:08.604437 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c4b0f0-a61a-44c4-a7c1-eb0d131ba115 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:08.604637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb811cb5-9200-407d-8944-247aa4dd2ef8 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:08.605885 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c4b0f0-a61a-44c4-a7c1-eb0d131ba115 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:08.606751 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:08.651549 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:08.701997 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 39/82] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:26:08.702858 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb811cb5-9200-407d-8944-247aa4dd2ef8 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:08.709235 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 44/83] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:26:08.709235 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4b0f0-a61a-44c4-a7c1-eb0d131ba115 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:08.805516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4b0f0-a61a-44c4-a7c1-eb0d131ba115 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:08.806164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb811cb5-9200-407d-8944-247aa4dd2ef8 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:08.808387 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:08.839420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cefe54-f9cf-4815-abc8-37c212b86e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:08.840063 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cefe54-f9cf-4815-abc8-37c212b86e7b None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:08.840350 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cefe54-f9cf-4815-abc8-37c212b86e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:08.840627 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cefe54-f9cf-4815-abc8-37c212b86e7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:08.842526 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee987a97-2785-45c9-be49-38cfb31798e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:08.843376 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee987a97-2785-45c9-be49-38cfb31798e3 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:08.843715 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee987a97-2785-45c9-be49-38cfb31798e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:08.844059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee987a97-2785-45c9-be49-38cfb31798e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:08.845405 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee987a97-2785-45c9-be49-38cfb31798e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:08.847041 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee987a97-2785-45c9-be49-38cfb31798e3 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:08.848679 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 40/84] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:26:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:08.849438 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee987a97-2785-45c9-be49-38cfb31798e3 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:08.859306 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:08.911736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cefe54-f9cf-4815-abc8-37c212b86e7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:08.913207 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cefe54-f9cf-4815-abc8-37c212b86e7b None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:08.944984 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee987a97-2785-45c9-be49-38cfb31798e3 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:08.978288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7270e8e-e016-46cf-8437-36eaf206912c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:08.978978 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7270e8e-e016-46cf-8437-36eaf206912c None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:08.979263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7270e8e-e016-46cf-8437-36eaf206912c None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:08.979540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7270e8e-e016-46cf-8437-36eaf206912c None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:08.980285 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7270e8e-e016-46cf-8437-36eaf206912c None admin] RBAC: Adding query filter params (name=member) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:08.980592 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7270e8e-e016-46cf-8437-36eaf206912c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:08.982847 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7270e8e-e016-46cf-8437-36eaf206912c None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:08.991010 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 41/85] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:08.991799 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7270e8e-e016-46cf-8437-36eaf206912c None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:09.015478 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 45/86] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:26:09.016251 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cefe54-f9cf-4815-abc8-37c212b86e7b None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:09.023573 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:09.062208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:09.093066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7270e8e-e016-46cf-8437-36eaf206912c None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:09.114689 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cefe54-f9cf-4815-abc8-37c212b86e7b None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:09.126582 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68699e-fdec-4305-894a-e5cb5c67b848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:09.127438 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68699e-fdec-4305-894a-e5cb5c67b848 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:09.127720 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68699e-fdec-4305-894a-e5cb5c67b848 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:09.127992 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68699e-fdec-4305-894a-e5cb5c67b848 None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:09.129236 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68699e-fdec-4305-894a-e5cb5c67b848 None admin] RBAC: Authorizing `identity:get_project(project_id=d1f168a334d74b6bb18fa28c3f9774d1)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:09.130683 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68699e-fdec-4305-894a-e5cb5c67b848 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:09.132320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 42/87] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:08 2022] GET /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:09.133049 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68699e-fdec-4305-894a-e5cb5c67b848 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:09.148444 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f15d3-72f0-443a-b7ce-b61226d39b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:09.149305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f15d3-72f0-443a-b7ce-b61226d39b32 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:09.149590 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f15d3-72f0-443a-b7ce-b61226d39b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:09.149873 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f15d3-72f0-443a-b7ce-b61226d39b32 None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:09.151081 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6f15d3-72f0-443a-b7ce-b61226d39b32 None admin] RBAC: Authorizing `identity:get_project(project_id=d1f168a334d74b6bb18fa28c3f9774d1)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:09.152544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6f15d3-72f0-443a-b7ce-b61226d39b32 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:09.154361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 46/88] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:09 2022] GET /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:09.155145 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f15d3-72f0-443a-b7ce-b61226d39b32 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:09.232427 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:09.235148 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68699e-fdec-4305-894a-e5cb5c67b848 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:09.255419 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f15d3-72f0-443a-b7ce-b61226d39b32 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:09.269415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:09.269534 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8550428-ace6-4e89-b53b-63eb7c484e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:09.270188 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8550428-ace6-4e89-b53b-63eb7c484e0c None admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:09.270507 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8550428-ace6-4e89-b53b-63eb7c484e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:09.270791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8550428-ace6-4e89-b53b-63eb7c484e0c None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:09.271510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8550428-ace6-4e89-b53b-63eb7c484e0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:09.272948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8550428-ace6-4e89-b53b-63eb7c484e0c None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:09.289971 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e1e14-579a-4952-a1fe-76cdecc6b416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:09.290871 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e1e14-579a-4952-a1fe-76cdecc6b416 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:09.291181 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e1e14-579a-4952-a1fe-76cdecc6b416 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:09.291492 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e1e14-579a-4952-a1fe-76cdecc6b416 None admin] PATH_INFO: `/v3/users/69a2de33463a462ca1305bf7bea04a37` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:09.292677 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e1e14-579a-4952-a1fe-76cdecc6b416 None admin] RBAC: Authorizing `identity:get_user(user_id=69a2de33463a462ca1305bf7bea04a37)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:09.294114 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e1e14-579a-4952-a1fe-76cdecc6b416 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:09.296281 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 47/89] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:09 2022] GET /identity/v3/users/69a2de33463a462ca1305bf7bea04a37 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:09.297033 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e1e14-579a-4952-a1fe-76cdecc6b416 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:09.378959 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 43/90] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Jun 27 18:26:09 2022] POST /identity/v3/users => generated 289 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:26:09.379776 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8550428-ace6-4e89-b53b-63eb7c484e0c None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:09.396392 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e1e14-579a-4952-a1fe-76cdecc6b416 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:09.436686 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc508e-c2de-43e7-88b2-afbd4ee8b906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:09.437569 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc508e-c2de-43e7-88b2-afbd4ee8b906 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:09.437852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc508e-c2de-43e7-88b2-afbd4ee8b906 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:09.438127 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc508e-c2de-43e7-88b2-afbd4ee8b906 None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:09.439449 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc508e-c2de-43e7-88b2-afbd4ee8b906 None admin] RBAC: Authorizing `identity:get_project(project_id=d1f168a334d74b6bb18fa28c3f9774d1)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:09.440922 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc508e-c2de-43e7-88b2-afbd4ee8b906 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:09.442959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 48/91] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:09 2022] GET /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:09.448472 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc508e-c2de-43e7-88b2-afbd4ee8b906 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:09.496507 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8550428-ace6-4e89-b53b-63eb7c484e0c None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:09.505048 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:09.537183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6390cb4-794c-4e0e-b87c-39c4ba567cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:09.537773 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6390cb4-794c-4e0e-b87c-39c4ba567cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:09.538064 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6390cb4-794c-4e0e-b87c-39c4ba567cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:09.538343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6390cb4-794c-4e0e-b87c-39c4ba567cf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:09.540092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6390cb4-794c-4e0e-b87c-39c4ba567cf0 None admin] RBAC: Adding query filter params (role.id=b45aa1c36d974fe79c809afbfcd01945, scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1, user.id=69a2de33463a462ca1305bf7bea04a37) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:09.540092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6390cb4-794c-4e0e-b87c-39c4ba567cf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:09.540976 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6390cb4-794c-4e0e-b87c-39c4ba567cf0 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:09.550048 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 44/92] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:26:09 2022] GET /identity/v3/role_assignments?user.id=69a2de33463a462ca1305bf7bea04a37&scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1&role.id=b45aa1c36d974fe79c809afbfcd01945 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:09.550922 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6390cb4-794c-4e0e-b87c-39c4ba567cf0 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:09.565638 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc508e-c2de-43e7-88b2-afbd4ee8b906 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:09.577400 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:09.614616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb74581-3fbd-4034-9872-8d296d17f43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:09.615310 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb74581-3fbd-4034-9872-8d296d17f43c None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:09.615603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb74581-3fbd-4034-9872-8d296d17f43c None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:09.615918 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb74581-3fbd-4034-9872-8d296d17f43c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:09.618983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb74581-3fbd-4034-9872-8d296d17f43c None admin] RBAC: Adding query filter params (role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f, scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1, user.id=3d0ddf436e7443679ab397306fca9e90) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:09.619313 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb74581-3fbd-4034-9872-8d296d17f43c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:09.620819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb74581-3fbd-4034-9872-8d296d17f43c None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:09.629662 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 49/93] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:26:09 2022] GET /identity/v3/role_assignments?user.id=3d0ddf436e7443679ab397306fca9e90&scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1&role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:09.630519 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb74581-3fbd-4034-9872-8d296d17f43c None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:09.696779 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6390cb4-794c-4e0e-b87c-39c4ba567cf0 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:09.751612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3959002-4dd6-4656-91b8-f8fe88811a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:09.753115 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3959002-4dd6-4656-91b8-f8fe88811a92 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:09.753703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3959002-4dd6-4656-91b8-f8fe88811a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:09.754377 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3959002-4dd6-4656-91b8-f8fe88811a92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:09.777689 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb74581-3fbd-4034-9872-8d296d17f43c None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:09.796879 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:09.822046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7839b0-7b66-4b81-bfd6-bbde0df07627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:09.822742 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7839b0-7b66-4b81-bfd6-bbde0df07627 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:09.823027 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7839b0-7b66-4b81-bfd6-bbde0df07627 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:09.823307 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7839b0-7b66-4b81-bfd6-bbde0df07627 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:09.824159 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7839b0-7b66-4b81-bfd6-bbde0df07627 None admin] RBAC: Adding query filter params (role.id=44d7d6bfe7eb441bbc6a301a5b6784ad, scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1, user.id=69a2de33463a462ca1305bf7bea04a37) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:09.824475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7839b0-7b66-4b81-bfd6-bbde0df07627 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:09.826609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7839b0-7b66-4b81-bfd6-bbde0df07627 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:09.832656 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 50/94] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:26:09 2022] GET /identity/v3/role_assignments?user.id=69a2de33463a462ca1305bf7bea04a37&scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1&role.id=44d7d6bfe7eb441bbc6a301a5b6784ad => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:09.852679 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3959002-4dd6-4656-91b8-f8fe88811a92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:09.854709 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3959002-4dd6-4656-91b8-f8fe88811a92 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:09.881325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:09.988921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 45/95] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:26:09.994487 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7839b0-7b66-4b81-bfd6-bbde0df07627 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:10.138913 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7839b0-7b66-4b81-bfd6-bbde0df07627 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:10.212896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf610771-51c2-4773-84c0-4b5873b1cc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:10.213772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf610771-51c2-4773-84c0-4b5873b1cc51 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:10.214055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf610771-51c2-4773-84c0-4b5873b1cc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:10.214332 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf610771-51c2-4773-84c0-4b5873b1cc51 None admin] PATH_INFO: `/v3/projects/0ee84742369a4858b3d2514ebfd17159` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:10.215621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf610771-51c2-4773-84c0-4b5873b1cc51 None admin] RBAC: Authorizing `identity:get_project(project_id=0ee84742369a4858b3d2514ebfd17159)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:10.217865 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf610771-51c2-4773-84c0-4b5873b1cc51 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:10.219661 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 51/96] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:09 2022] GET /identity/v3/projects/0ee84742369a4858b3d2514ebfd17159 => generated 312 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:10.225553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3959002-4dd6-4656-91b8-f8fe88811a92 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:10.381214 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3959002-4dd6-4656-91b8-f8fe88811a92 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:10.409681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:10.428306 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc26f5-7ab3-4d18-81f8-061493ec27dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:10.429316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc26f5-7ab3-4d18-81f8-061493ec27dd None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:10.429807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc26f5-7ab3-4d18-81f8-061493ec27dd None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:10.430283 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc26f5-7ab3-4d18-81f8-061493ec27dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:10.431352 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc26f5-7ab3-4d18-81f8-061493ec27dd None admin] RBAC: Adding query filter params (role.id=b45aa1c36d974fe79c809afbfcd01945, scope.project.id=0ee84742369a4858b3d2514ebfd17159, user.id=69a2de33463a462ca1305bf7bea04a37) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:10.431890 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc26f5-7ab3-4d18-81f8-061493ec27dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:10.433546 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc26f5-7ab3-4d18-81f8-061493ec27dd None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:10.442825 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 46/97] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:26:10 2022] GET /identity/v3/role_assignments?user.id=69a2de33463a462ca1305bf7bea04a37&scope.project.id=0ee84742369a4858b3d2514ebfd17159&role.id=b45aa1c36d974fe79c809afbfcd01945 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:12.313648 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c64bb-5983-4b35-9cb2-78ac0c12a927 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:12.314156 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c64bb-5983-4b35-9cb2-78ac0c12a927 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:12.314231 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c64bb-5983-4b35-9cb2-78ac0c12a927 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:12.315728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 52/98] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:12.332902 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ef93a-e72d-4284-9570-d0602c0ee04b None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:12.333808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ef93a-e72d-4284-9570-d0602c0ee04b None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:12.334294 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ef93a-e72d-4284-9570-d0602c0ee04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:12.354521 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-aa9ef93a-e72d-4284-9570-d0602c0ee04b None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:12.356063 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-aa9ef93a-e72d-4284-9570-d0602c0ee04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:12.440024 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 47/99] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:12.474733 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c64bb-5983-4b35-9cb2-78ac0c12a927 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:12.532580 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2653f03-f727-4a21-b111-10f96f15038c None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:12.533212 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2653f03-f727-4a21-b111-10f96f15038c None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:12.533695 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2653f03-f727-4a21-b111-10f96f15038c None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:12.535583 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 48/100] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:12.540493 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e15b5-81bd-4172-ba82-203589b89187 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:12.541326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e15b5-81bd-4172-ba82-203589b89187 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:12.541812 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e15b5-81bd-4172-ba82-203589b89187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:12.544208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:12.569196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-273e15b5-81bd-4172-ba82-203589b89187 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:12.574937 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-273e15b5-81bd-4172-ba82-203589b89187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:12.624567 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c64bb-5983-4b35-9cb2-78ac0c12a927 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:12.696496 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba5b0b-6594-4c6f-9e9d-a8a9e46f8914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:12.697300 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba5b0b-6594-4c6f-9e9d-a8a9e46f8914 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:12.697590 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba5b0b-6594-4c6f-9e9d-a8a9e46f8914 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:12.697872 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba5b0b-6594-4c6f-9e9d-a8a9e46f8914 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:12.705233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 49/101] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:12.709221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be17e9-d188-4a53-8539-88d22a688a15 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:12.709517 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be17e9-d188-4a53-8539-88d22a688a15 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:12.709806 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be17e9-d188-4a53-8539-88d22a688a15 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:12.711168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 50/102] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:12.718362 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96077c0e-6004-4a5b-93c1-360a42a8b615 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:12.718676 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96077c0e-6004-4a5b-93c1-360a42a8b615 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:12.718960 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96077c0e-6004-4a5b-93c1-360a42a8b615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:12.726131 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba5b0b-6594-4c6f-9e9d-a8a9e46f8914 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:12.735259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-b0ba5b0b-6594-4c6f-9e9d-a8a9e46f8914 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:12.735259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:12.735259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:12.735259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:12.735259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:12.735259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:12.735259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:12.735259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:12.735259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:12.735259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:12.735259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:12.735259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:12.735259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:12.735259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:12.735259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:12.735259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:12.736112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:12.736112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:12.736112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:12.736112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:12.736112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:12.736112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:12.736112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:12.736112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:12.736112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:12.736112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:12.736112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:12.736112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:12.736112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:12.736112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:12.736112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:12.736112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:12.736862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:12.736862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:12.736862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:12.736862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:12.736862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:12.736862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:12.736862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:12.736862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:12.736862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:12.736862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:12.736862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:12.736862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:12.736862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 53/103] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:12.742643 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba5b0b-6594-4c6f-9e9d-a8a9e46f8914 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:12.775421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-96077c0e-6004-4a5b-93c1-360a42a8b615 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:12.776623 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-96077c0e-6004-4a5b-93c1-360a42a8b615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:12.844940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:12.858305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba5b0b-6594-4c6f-9e9d-a8a9e46f8914 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:12.866118 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:12.881400 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 51/104] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:26:12.882956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96077c0e-6004-4a5b-93c1-360a42a8b615 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:12.895713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955c581-f9d7-4c63-8995-97d6cbca7505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:12.896819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955c581-f9d7-4c63-8995-97d6cbca7505 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:12.897322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955c581-f9d7-4c63-8995-97d6cbca7505 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:12.897808 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955c581-f9d7-4c63-8995-97d6cbca7505 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:12.911159 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955c581-f9d7-4c63-8995-97d6cbca7505 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:12.922739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-9955c581-f9d7-4c63-8995-97d6cbca7505 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:26:12.922739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:12.922739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:12.922739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:12.922739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:12.922739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:12.922739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:12.922739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:12.922739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:12.922739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:12.922739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:12.922739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:12.922739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:12.922739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:12.922739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:12.922739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:12.923617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:12.923617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:12.923617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:12.923617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:12.923617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:12.923617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:12.923617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:12.923617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:12.923617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:12.923617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:12.923617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:12.923617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:12.923617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:12.923617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:12.923617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:12.923617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:12.924413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:12.924413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:12.924413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:12.924413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:12.924413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:12.924413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:12.924413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:12.924413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:12.924413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:12.924413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:12.924413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:26:12.924413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:12.927715 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 54/105] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:26:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:26:12.933723 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49578b2c-dcf9-4ecf-81c7-850d6b4e2a3b None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:12.934576 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49578b2c-dcf9-4ecf-81c7-850d6b4e2a3b None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:12.934576 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49578b2c-dcf9-4ecf-81c7-850d6b4e2a3b None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:12.936418 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 55/106] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:12.938917 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49578b2c-dcf9-4ecf-81c7-850d6b4e2a3b None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:13.000689 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96077c0e-6004-4a5b-93c1-360a42a8b615 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:13.036625 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e9032-8a36-4682-a906-f733bab87f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:13.037717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e9032-8a36-4682-a906-f733bab87f6d None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:13.038223 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e9032-8a36-4682-a906-f733bab87f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:13.038724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e9032-8a36-4682-a906-f733bab87f6d None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:13.039678 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e9032-8a36-4682-a906-f733bab87f6d None admin] RBAC: Adding query filter params (name=member) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:13.040179 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e9032-8a36-4682-a906-f733bab87f6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:13.041822 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e9032-8a36-4682-a906-f733bab87f6d None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:13.051608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 52/107] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:13.052729 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e9032-8a36-4682-a906-f733bab87f6d None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:13.058697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49578b2c-dcf9-4ecf-81c7-850d6b4e2a3b None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:13.090275 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:13.101727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8121e-ced6-4926-b1bc-fd1467d9cd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:13.101727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8121e-ced6-4926-b1bc-fd1467d9cd4c None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:13.101727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8121e-ced6-4926-b1bc-fd1467d9cd4c None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:13.101727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8121e-ced6-4926-b1bc-fd1467d9cd4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:13.102081 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8121e-ced6-4926-b1bc-fd1467d9cd4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:13.102514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8121e-ced6-4926-b1bc-fd1467d9cd4c None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:13.104371 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 56/108] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:26:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:13.105340 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf11166-9479-487c-b6aa-800e1c1ddaf3 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:13.105598 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf11166-9479-487c-b6aa-800e1c1ddaf3 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:13.105935 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf11166-9479-487c-b6aa-800e1c1ddaf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:13.125092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-7bf11166-9479-487c-b6aa-800e1c1ddaf3 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:13.126335 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-7bf11166-9479-487c-b6aa-800e1c1ddaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:13.156774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:13.161029 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e9032-8a36-4682-a906-f733bab87f6d None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:13.196327 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd7ec8-42b5-49b1-8b66-539b9ea57f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:13.196981 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd7ec8-42b5-49b1-8b66-539b9ea57f9f None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:13.197263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd7ec8-42b5-49b1-8b66-539b9ea57f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:13.197536 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd7ec8-42b5-49b1-8b66-539b9ea57f9f None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:13.198278 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd7ec8-42b5-49b1-8b66-539b9ea57f9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:13.198610 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd7ec8-42b5-49b1-8b66-539b9ea57f9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:13.199997 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd7ec8-42b5-49b1-8b66-539b9ea57f9f None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:13.209214 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 53/109] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:13.209214 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd7ec8-42b5-49b1-8b66-539b9ea57f9f None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:13.211160 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 57/110] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:13.212452 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb36648-eb9f-42eb-81d3-9dbb63a08293 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:13.212747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb36648-eb9f-42eb-81d3-9dbb63a08293 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:13.213038 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb36648-eb9f-42eb-81d3-9dbb63a08293 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:13.214360 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 58/111] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:13.215082 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb36648-eb9f-42eb-81d3-9dbb63a08293 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:13.397339 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:13.399915 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb36648-eb9f-42eb-81d3-9dbb63a08293 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:13.400895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd7ec8-42b5-49b1-8b66-539b9ea57f9f None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:13.436860 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416b6f5-199f-444c-920c-120a6551296c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:13.437517 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416b6f5-199f-444c-920c-120a6551296c None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:13.437806 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416b6f5-199f-444c-920c-120a6551296c None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:13.438086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416b6f5-199f-444c-920c-120a6551296c None admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:13.438831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416b6f5-199f-444c-920c-120a6551296c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:13.439316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec4b96-27c8-4772-9362-0184e25f5503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:13.440282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416b6f5-199f-444c-920c-120a6551296c None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:13.440606 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec4b96-27c8-4772-9362-0184e25f5503 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:13.441072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec4b96-27c8-4772-9362-0184e25f5503 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:13.441640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec4b96-27c8-4772-9362-0184e25f5503 None admin] PATH_INFO: `/v3/users/69a2de33463a462ca1305bf7bea04a37` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:13.443148 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec4b96-27c8-4772-9362-0184e25f5503 None admin] RBAC: Authorizing `identity:get_user(user_id=69a2de33463a462ca1305bf7bea04a37)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:13.444937 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec4b96-27c8-4772-9362-0184e25f5503 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:13.446816 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 54/112] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:13 2022] GET /identity/v3/users/69a2de33463a462ca1305bf7bea04a37 => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:13.447756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec4b96-27c8-4772-9362-0184e25f5503 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:13.522809 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 59/113] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Jun 27 18:26:13 2022] POST /identity/v3/projects => generated 302 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:26:13.536068 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaee25a-9611-4779-aa01-f6fa504fd216 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:13.536402 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaee25a-9611-4779-aa01-f6fa504fd216 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:13.536633 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaee25a-9611-4779-aa01-f6fa504fd216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:13.561339 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec4b96-27c8-4772-9362-0184e25f5503 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:13.568406 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-ecaee25a-9611-4779-aa01-f6fa504fd216 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:13.570077 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ecaee25a-9611-4779-aa01-f6fa504fd216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:13.583849 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:13.613497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64e2da-80b6-41e9-bd83-67e456d41dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:13.614458 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64e2da-80b6-41e9-bd83-67e456d41dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:13.614824 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64e2da-80b6-41e9-bd83-67e456d41dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:13.615113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a64e2da-80b6-41e9-bd83-67e456d41dc5 None admin] PATH_INFO: `/v3/users/3d0ddf436e7443679ab397306fca9e90` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:13.616523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a64e2da-80b6-41e9-bd83-67e456d41dc5 None admin] RBAC: Authorizing `identity:get_user(user_id=3d0ddf436e7443679ab397306fca9e90)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:13.618107 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a64e2da-80b6-41e9-bd83-67e456d41dc5 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:13.625757 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 55/114] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:13 2022] GET /identity/v3/users/3d0ddf436e7443679ab397306fca9e90 => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:13.625757 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64e2da-80b6-41e9-bd83-67e456d41dc5 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:13.657228 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 60/115] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:13.658574 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaee25a-9611-4779-aa01-f6fa504fd216 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:13.666064 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:13.758808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a64e2da-80b6-41e9-bd83-67e456d41dc5 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:13.782159 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaee25a-9611-4779-aa01-f6fa504fd216 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:13.796290 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289b603-85d7-469f-89e0-3dc5576368d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:13.797098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289b603-85d7-469f-89e0-3dc5576368d3 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:13.797382 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289b603-85d7-469f-89e0-3dc5576368d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:13.797657 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4289b603-85d7-469f-89e0-3dc5576368d3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:13.808126 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4289b603-85d7-469f-89e0-3dc5576368d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:13.816774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-4289b603-85d7-469f-89e0-3dc5576368d3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:13.816774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:13.816774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:13.816774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:13.816774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:13.816774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:13.816774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:13.816774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:13.816774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:13.816774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:13.816774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:13.816774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:13.816774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:13.816774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:13.816774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:13.816774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:13.817655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:13.817655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:13.817655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:13.817655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:13.817655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:13.817655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:13.817655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:13.817655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:13.817655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:13.817655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:13.817655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:13.817655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:13.817655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:13.817655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:13.817655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:13.817655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:13.818837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:13.818837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:13.818837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:13.818837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:13.818837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:13.818837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:13.818837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:13.818837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:13.818837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:13.818837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:13.818837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:13.818837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:26:13.818837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 56/116] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Jun 27 18:26:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:13.819513 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289b603-85d7-469f-89e0-3dc5576368d3 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:13.824687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0f2a1-e627-4a02-9acf-149b53caf937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:13.825611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0f2a1-e627-4a02-9acf-149b53caf937 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:13.825920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0f2a1-e627-4a02-9acf-149b53caf937 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:13.826182 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0f2a1-e627-4a02-9acf-149b53caf937 None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:13.827444 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0f2a1-e627-4a02-9acf-149b53caf937 None admin] RBAC: Authorizing `identity:get_project(project_id=d1f168a334d74b6bb18fa28c3f9774d1)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:13.828914 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0f2a1-e627-4a02-9acf-149b53caf937 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:13.830683 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 61/117] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:13 2022] GET /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:13.831848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e8e53-136e-4327-a193-42d0864ed9de None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:13.832139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e8e53-136e-4327-a193-42d0864ed9de None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:13.832420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e8e53-136e-4327-a193-42d0864ed9de None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:13.833650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 62/118] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:13.838291 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e8e53-136e-4327-a193-42d0864ed9de None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:13.859347 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:13.922056 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:13.929277 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4289b603-85d7-469f-89e0-3dc5576368d3 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:13.950404 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e8e53-136e-4327-a193-42d0864ed9de None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:13.967160 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443aaac-1265-4a76-a5e9-79c1073bd94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:13.968056 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443aaac-1265-4a76-a5e9-79c1073bd94d None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:13.968346 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443aaac-1265-4a76-a5e9-79c1073bd94d None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:13.968627 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443aaac-1265-4a76-a5e9-79c1073bd94d None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:13.969834 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443aaac-1265-4a76-a5e9-79c1073bd94d None admin] RBAC: Authorizing `identity:get_project(project_id=d1f168a334d74b6bb18fa28c3f9774d1)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:13.971321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443aaac-1265-4a76-a5e9-79c1073bd94d None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:13.973349 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 57/119] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:13 2022] GET /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:13.974131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443aaac-1265-4a76-a5e9-79c1073bd94d None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:13.994138 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58921a-2d77-42b3-9dba-444011d90de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:13.995499 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58921a-2d77-42b3-9dba-444011d90de4 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:13.995965 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58921a-2d77-42b3-9dba-444011d90de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:13.996566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58921a-2d77-42b3-9dba-444011d90de4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:14.011208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de58921a-2d77-42b3-9dba-444011d90de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:14.022094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-de58921a-2d77-42b3-9dba-444011d90de4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 18:26:14.022094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:14.022094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:14.022094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:14.022094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:14.022094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:14.022094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:14.022094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:14.022094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:14.022094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:14.022094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:14.022094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:14.022094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:14.022094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:14.022094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:14.022094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:14.022987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:14.022987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:14.022987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:14.022987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:14.022987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:14.022987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:14.022987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:14.022987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:14.022987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:14.022987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:14.022987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:14.022987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:14.022987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:14.022987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:14.022987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:14.022987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:14.023743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:14.023743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:14.023743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:14.023743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:14.023743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:14.023743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:14.023743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:14.023743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:14.023743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:14.023743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:14.023743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 18:26:14.023743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:14.026271 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 63/120] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:14.027258 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58921a-2d77-42b3-9dba-444011d90de4 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:14.093302 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443aaac-1265-4a76-a5e9-79c1073bd94d None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:14.108584 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:14.133661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8d91b-7c82-4f2c-9aca-cf966d72a4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:14.134619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8d91b-7c82-4f2c-9aca-cf966d72a4fc None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:14.135029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58921a-2d77-42b3-9dba-444011d90de4 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:14.135226 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8d91b-7c82-4f2c-9aca-cf966d72a4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:14.135638 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8d91b-7c82-4f2c-9aca-cf966d72a4fc None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:14.136688 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8d91b-7c82-4f2c-9aca-cf966d72a4fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:14.138003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8d91b-7c82-4f2c-9aca-cf966d72a4fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:14.139586 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8d91b-7c82-4f2c-9aca-cf966d72a4fc None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:14.148983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 58/121] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:26:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:14.151691 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7c743-c81b-4520-ac45-d219bc46a03d None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:14.151973 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7c743-c81b-4520-ac45-d219bc46a03d None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:14.152249 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7c743-c81b-4520-ac45-d219bc46a03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:14.171013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-2ff7c743-c81b-4520-ac45-d219bc46a03d None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:14.172176 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-2ff7c743-c81b-4520-ac45-d219bc46a03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:14.176068 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b816b45-5e78-4034-9028-298a11892738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:14.176958 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b816b45-5e78-4034-9028-298a11892738 None admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:14.177246 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b816b45-5e78-4034-9028-298a11892738 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:14.177520 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b816b45-5e78-4034-9028-298a11892738 None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1/users/69a2de33463a462ca1305bf7bea04a37/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:14.179921 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b816b45-5e78-4034-9028-298a11892738 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1f168a334d74b6bb18fa28c3f9774d1, user_id=69a2de33463a462ca1305bf7bea04a37, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:14.181724 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b816b45-5e78-4034-9028-298a11892738 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:14.190038 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:14.239323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 59/122] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:14.240036 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7c743-c81b-4520-ac45-d219bc46a03d None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:14.259641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 64/123] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:26:14 2022] PUT /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1/users/69a2de33463a462ca1305bf7bea04a37/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:26:14.261509 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b816b45-5e78-4034-9028-298a11892738 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:14.378362 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7c743-c81b-4520-ac45-d219bc46a03d None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:14.393860 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b816b45-5e78-4034-9028-298a11892738 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:14.431021 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59bb47-d6d0-447a-b6fc-89a67d9034ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:14.431780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59bb47-d6d0-447a-b6fc-89a67d9034ac None admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:14.432066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59bb47-d6d0-447a-b6fc-89a67d9034ac None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:14.432345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59bb47-d6d0-447a-b6fc-89a67d9034ac None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1/users/3d0ddf436e7443679ab397306fca9e90/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:14.434335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59bb47-d6d0-447a-b6fc-89a67d9034ac None admin] RBAC: Authorizing `identity:create_grant(project_id=d1f168a334d74b6bb18fa28c3f9774d1, user_id=3d0ddf436e7443679ab397306fca9e90, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:14.436202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59bb47-d6d0-447a-b6fc-89a67d9034ac None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:14.440606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c888c-6afd-4087-a222-74fb710866ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:14.441287 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c888c-6afd-4087-a222-74fb710866ac None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:14.441577 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c888c-6afd-4087-a222-74fb710866ac None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:14.441858 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c888c-6afd-4087-a222-74fb710866ac None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:14.442657 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c888c-6afd-4087-a222-74fb710866ac None admin] RBAC: Adding query filter params (name=reader) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:14.442970 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c888c-6afd-4087-a222-74fb710866ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:14.444359 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c888c-6afd-4087-a222-74fb710866ac None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:14.445833 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:14.454488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 65/124] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:14.455148 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c888c-6afd-4087-a222-74fb710866ac None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:14.534312 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 60/125] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:26:14 2022] PUT /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1/users/3d0ddf436e7443679ab397306fca9e90/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:26:14.535182 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59bb47-d6d0-447a-b6fc-89a67d9034ac None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:14.556118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:14.586991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c888c-6afd-4087-a222-74fb710866ac None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:14.667207 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad07c62c-d8fd-4556-a2a8-93d010de262e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:14.668278 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07c62c-d8fd-4556-a2a8-93d010de262e None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:14.668657 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07c62c-d8fd-4556-a2a8-93d010de262e None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:14.669030 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07c62c-d8fd-4556-a2a8-93d010de262e None admin] PATH_INFO: `/v3/users/69a2de33463a462ca1305bf7bea04a37` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:14.671524 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad07c62c-d8fd-4556-a2a8-93d010de262e None admin] RBAC: Authorizing `identity:get_user(user_id=69a2de33463a462ca1305bf7bea04a37)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:14.673151 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad07c62c-d8fd-4556-a2a8-93d010de262e None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:14.675018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 66/126] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:14 2022] GET /identity/v3/users/69a2de33463a462ca1305bf7bea04a37 => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:14.675910 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad07c62c-d8fd-4556-a2a8-93d010de262e None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:14.680655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59bb47-d6d0-447a-b6fc-89a67d9034ac None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:14.729106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509da52-90ba-4ae7-aa19-ca292645a37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:14.730133 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509da52-90ba-4ae7-aa19-ca292645a37a None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:14.730613 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509da52-90ba-4ae7-aa19-ca292645a37a None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:14.731035 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509da52-90ba-4ae7-aa19-ca292645a37a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:14.757318 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3509da52-90ba-4ae7-aa19-ca292645a37a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:14.765364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-3509da52-90ba-4ae7-aa19-ca292645a37a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:14.765364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:14.765364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:14.765364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:14.765364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:14.765364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:14.765364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:14.765364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:14.765364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:14.765364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:14.765364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:14.765364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:14.765364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:14.765364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:14.765364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:14.765364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:14.766612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:14.766612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:14.766612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:14.766612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:14.766612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:14.766612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:14.766612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:14.766612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:14.766612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:14.766612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:14.766612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:14.766612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:14.766612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:14.766612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:14.766612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:14.766612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:14.771808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:14.771808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:14.771808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:14.771808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:14.771808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:14.771808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:14.771808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:14.771808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:14.771808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:14.771808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:14.771808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:14.771808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:26:14.771808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 61/127] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:14.771808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509da52-90ba-4ae7-aa19-ca292645a37a None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:14.807631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:14.859924 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:14.861355 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad07c62c-d8fd-4556-a2a8-93d010de262e None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:14.885504 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509da52-90ba-4ae7-aa19-ca292645a37a None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:14.908497 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53ae2a-7d27-4494-a651-e159dd1bab01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:14.910034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53ae2a-7d27-4494-a651-e159dd1bab01 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:14.910751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53ae2a-7d27-4494-a651-e159dd1bab01 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:14.911707 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53ae2a-7d27-4494-a651-e159dd1bab01 None admin] PATH_INFO: `/v3/users/e50cef24722a4615ad1077c26e19c162` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:14.928897 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53ae2a-7d27-4494-a651-e159dd1bab01 None admin] RBAC: Authorizing `identity:get_user(user_id=e50cef24722a4615ad1077c26e19c162)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:14.929084 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8bb09-4ad4-4013-b5f3-787ca0ca7703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:14.929761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8bb09-4ad4-4013-b5f3-787ca0ca7703 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:14.930042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8bb09-4ad4-4013-b5f3-787ca0ca7703 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:14.930318 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8bb09-4ad4-4013-b5f3-787ca0ca7703 None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:14.931498 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53ae2a-7d27-4494-a651-e159dd1bab01 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:14.931609 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8bb09-4ad4-4013-b5f3-787ca0ca7703 None admin] RBAC: Authorizing `identity:get_project(project_id=d1f168a334d74b6bb18fa28c3f9774d1)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:14.932967 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8bb09-4ad4-4013-b5f3-787ca0ca7703 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:14.933829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 67/128] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:14 2022] GET /identity/v3/users/e50cef24722a4615ad1077c26e19c162 => generated 289 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:14.935229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 62/129] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:14 2022] GET /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:14.935710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53ae2a-7d27-4494-a651-e159dd1bab01 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:14.941549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8bb09-4ad4-4013-b5f3-787ca0ca7703 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:15.066221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8bb09-4ad4-4013-b5f3-787ca0ca7703 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:15.120691 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aebecf-8980-47e2-8725-59008f10c904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:15.121340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aebecf-8980-47e2-8725-59008f10c904 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:15.121638 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aebecf-8980-47e2-8725-59008f10c904 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:15.121907 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aebecf-8980-47e2-8725-59008f10c904 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:15.142172 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53ae2a-7d27-4494-a651-e159dd1bab01 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:15.150721 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:15.177907 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:15.206593 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f8bd2-beba-4995-b962-19ee03d54043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:15.207278 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f8bd2-beba-4995-b962-19ee03d54043 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:15.207570 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f8bd2-beba-4995-b962-19ee03d54043 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:15.207853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f8bd2-beba-4995-b962-19ee03d54043 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:15.208622 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f8bd2-beba-4995-b962-19ee03d54043 None admin] RBAC: Adding query filter params (name=member) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:15.208932 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f8bd2-beba-4995-b962-19ee03d54043 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:15.210337 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f8bd2-beba-4995-b962-19ee03d54043 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:15.230760 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aebecf-8980-47e2-8725-59008f10c904 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:15.232336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 68/130] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:15.233285 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f8bd2-beba-4995-b962-19ee03d54043 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:15.237519 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aebecf-8980-47e2-8725-59008f10c904 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:15.370775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 63/131] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:26:15.371709 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aebecf-8980-47e2-8725-59008f10c904 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:15.404462 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f8bd2-beba-4995-b962-19ee03d54043 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:15.416131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:15.446308 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e93485-be9c-49f2-b8bb-214b3452e905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:15.447240 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e93485-be9c-49f2-b8bb-214b3452e905 None admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:15.447525 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e93485-be9c-49f2-b8bb-214b3452e905 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:15.447807 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e93485-be9c-49f2-b8bb-214b3452e905 None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1/users/69a2de33463a462ca1305bf7bea04a37/roles/44d7d6bfe7eb441bbc6a301a5b6784ad` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:15.452673 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e93485-be9c-49f2-b8bb-214b3452e905 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1f168a334d74b6bb18fa28c3f9774d1, user_id=69a2de33463a462ca1305bf7bea04a37, role_id=44d7d6bfe7eb441bbc6a301a5b6784ad)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:15.454132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e93485-be9c-49f2-b8bb-214b3452e905 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:15.499717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aebecf-8980-47e2-8725-59008f10c904 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:15.539662 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 69/132] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:26:15 2022] PUT /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1/users/69a2de33463a462ca1305bf7bea04a37/roles/44d7d6bfe7eb441bbc6a301a5b6784ad => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:26:15.541407 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e0c565-a636-4e5a-8eff-af8c7fd01cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:15.542306 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0c565-a636-4e5a-8eff-af8c7fd01cae None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:15.542617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0c565-a636-4e5a-8eff-af8c7fd01cae None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:15.542917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0c565-a636-4e5a-8eff-af8c7fd01cae None admin] PATH_INFO: `/v3/users/69a2de33463a462ca1305bf7bea04a37` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:15.543405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e93485-be9c-49f2-b8bb-214b3452e905 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:15.545331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e0c565-a636-4e5a-8eff-af8c7fd01cae None admin] RBAC: Authorizing `identity:get_user(user_id=69a2de33463a462ca1305bf7bea04a37)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:15.547986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e0c565-a636-4e5a-8eff-af8c7fd01cae None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:15.550134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 64/133] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:15 2022] GET /identity/v3/users/69a2de33463a462ca1305bf7bea04a37 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:15.561471 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e0c565-a636-4e5a-8eff-af8c7fd01cae None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:15.657468 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:15.790649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e93485-be9c-49f2-b8bb-214b3452e905 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:15.828889 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e0c565-a636-4e5a-8eff-af8c7fd01cae None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:15.844642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:15.854576 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17ae83-189b-4234-a6c9-9ce41dcfa3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:15.855447 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17ae83-189b-4234-a6c9-9ce41dcfa3af None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:15.858195 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17ae83-189b-4234-a6c9-9ce41dcfa3af None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:15.858195 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17ae83-189b-4234-a6c9-9ce41dcfa3af None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:15.858660 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e17ae83-189b-4234-a6c9-9ce41dcfa3af None admin] RBAC: Authorizing `identity:get_project(project_id=d1f168a334d74b6bb18fa28c3f9774d1)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:15.860213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e17ae83-189b-4234-a6c9-9ce41dcfa3af None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:15.861930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 70/134] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:15 2022] GET /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1 => generated 298 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:15.868461 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17ae83-189b-4234-a6c9-9ce41dcfa3af None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:15.896494 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcc271-098f-42b0-9ffa-1bc99b90743e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:15.898116 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcc271-098f-42b0-9ffa-1bc99b90743e None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:15.898427 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcc271-098f-42b0-9ffa-1bc99b90743e None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:15.898712 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcc271-098f-42b0-9ffa-1bc99b90743e None admin] PATH_INFO: `/v3/projects/0ee84742369a4858b3d2514ebfd17159` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:15.903095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffcc271-098f-42b0-9ffa-1bc99b90743e None admin] RBAC: Authorizing `identity:get_project(project_id=0ee84742369a4858b3d2514ebfd17159)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:15.905345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffcc271-098f-42b0-9ffa-1bc99b90743e None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:15.907731 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 65/135] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:15 2022] GET /identity/v3/projects/0ee84742369a4858b3d2514ebfd17159 => generated 312 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:15.908935 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec4de1-54bd-47be-83c6-d99358e9b944 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:15.909230 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec4de1-54bd-47be-83c6-d99358e9b944 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:15.909510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec4de1-54bd-47be-83c6-d99358e9b944 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:15.910889 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 66/136] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:15.917648 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cba13e-c4db-4ff9-8f2e-83e00c3a0258 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:15.917942 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cba13e-c4db-4ff9-8f2e-83e00c3a0258 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:15.918226 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cba13e-c4db-4ff9-8f2e-83e00c3a0258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:15.953406 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-a0cba13e-c4db-4ff9-8f2e-83e00c3a0258 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:15.954646 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-a0cba13e-c4db-4ff9-8f2e-83e00c3a0258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:16.067300 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 67/137] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:16.068145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cba13e-c4db-4ff9-8f2e-83e00c3a0258 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:16.082127 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17ae83-189b-4234-a6c9-9ce41dcfa3af None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:16.106359 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:16.151059 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac891c0-8eb7-4a44-bde3-252ee67ec60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:16.151725 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac891c0-8eb7-4a44-bde3-252ee67ec60d None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:16.152011 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac891c0-8eb7-4a44-bde3-252ee67ec60d None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:16.152293 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac891c0-8eb7-4a44-bde3-252ee67ec60d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:16.153136 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac891c0-8eb7-4a44-bde3-252ee67ec60d None admin] RBAC: Adding query filter params (role.id=b042cc08697840f5a9ae4066a6f0205c, scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1, user.id=e50cef24722a4615ad1077c26e19c162) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:16.153478 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac891c0-8eb7-4a44-bde3-252ee67ec60d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:16.154991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac891c0-8eb7-4a44-bde3-252ee67ec60d None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:16.167109 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 71/138] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:26:15 2022] GET /identity/v3/role_assignments?user.id=e50cef24722a4615ad1077c26e19c162&scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1&role.id=b042cc08697840f5a9ae4066a6f0205c => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:16.167858 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac891c0-8eb7-4a44-bde3-252ee67ec60d None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:16.274222 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cba13e-c4db-4ff9-8f2e-83e00c3a0258 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:16.297398 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:16.365019 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be6561-2207-49b5-91c8-25ce40a73ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:16.365928 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be6561-2207-49b5-91c8-25ce40a73ea5 None admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:16.366221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be6561-2207-49b5-91c8-25ce40a73ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:16.366530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be6561-2207-49b5-91c8-25ce40a73ea5 None admin] PATH_INFO: `/v3/projects/0ee84742369a4858b3d2514ebfd17159/users/69a2de33463a462ca1305bf7bea04a37/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:16.368624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be6561-2207-49b5-91c8-25ce40a73ea5 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ee84742369a4858b3d2514ebfd17159, user_id=69a2de33463a462ca1305bf7bea04a37, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:16.370087 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be6561-2207-49b5-91c8-25ce40a73ea5 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:16.388825 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac891c0-8eb7-4a44-bde3-252ee67ec60d None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:16.449938 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e90e6a-ec24-4b5a-8d25-62fa20fbabd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:16.450784 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e90e6a-ec24-4b5a-8d25-62fa20fbabd3 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:16.451079 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e90e6a-ec24-4b5a-8d25-62fa20fbabd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:16.451299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e90e6a-ec24-4b5a-8d25-62fa20fbabd3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:16.453947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e90e6a-ec24-4b5a-8d25-62fa20fbabd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:16.455485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e90e6a-ec24-4b5a-8d25-62fa20fbabd3 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:16.457167 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 72/139] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:26:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:16.462977 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e90e6a-ec24-4b5a-8d25-62fa20fbabd3 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:16.521862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:16.523438 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 68/140] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:26:16 2022] PUT /identity/v3/projects/0ee84742369a4858b3d2514ebfd17159/users/69a2de33463a462ca1305bf7bea04a37/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:26:16.597179 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e90e6a-ec24-4b5a-8d25-62fa20fbabd3 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:16.643004 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29238d-c835-4a8d-bad9-a9a1229a0d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:16.643682 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29238d-c835-4a8d-bad9-a9a1229a0d79 None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:16.644024 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29238d-c835-4a8d-bad9-a9a1229a0d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:16.644307 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29238d-c835-4a8d-bad9-a9a1229a0d79 None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:16.645030 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29238d-c835-4a8d-bad9-a9a1229a0d79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:16.646590 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29238d-c835-4a8d-bad9-a9a1229a0d79 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:16.683297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21abd9-44f9-4d50-b5d8-42358d67b4ec None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:16.683619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21abd9-44f9-4d50-b5d8-42358d67b4ec None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:16.683870 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21abd9-44f9-4d50-b5d8-42358d67b4ec None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:16.685423 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 69/141] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:26:16.691143 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77399d0-fd15-4567-9fc2-8b1778c9d58b None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:16.691479 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77399d0-fd15-4567-9fc2-8b1778c9d58b None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:16.691782 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77399d0-fd15-4567-9fc2-8b1778c9d58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:16.718403 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-e77399d0-fd15-4567-9fc2-8b1778c9d58b None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:16.719535 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-e77399d0-fd15-4567-9fc2-8b1778c9d58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:16.741411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 73/142] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Jun 27 18:26:16 2022] POST /identity/v3/users => generated 283 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:26:16.825867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 70/143] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:26:16.867361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29238d-c835-4a8d-bad9-a9a1229a0d79 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:17.013795 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:17.016805 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29238d-c835-4a8d-bad9-a9a1229a0d79 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:17.059412 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7d3a5-54f4-401b-ab3f-d63dec04e983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:17.060270 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7d3a5-54f4-401b-ab3f-d63dec04e983 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:17.060560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7d3a5-54f4-401b-ab3f-d63dec04e983 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:17.060831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7d3a5-54f4-401b-ab3f-d63dec04e983 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:17.073653 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db7d3a5-54f4-401b-ab3f-d63dec04e983 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:17.081708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-6db7d3a5-54f4-401b-ab3f-d63dec04e983 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:17.081708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:17.081708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:17.081708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:17.081708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:17.081708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:17.081708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:17.081708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:17.081708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:17.081708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:17.081708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:17.081708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:17.081708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:17.081708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:17.081708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:17.081708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:17.082621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:17.082621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:17.082621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:17.082621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:17.082621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:17.082621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:17.082621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:17.082621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:17.082621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:17.082621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:17.082621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:17.082621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:17.082621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:17.082621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:17.082621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:17.082621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:17.083420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:17.083420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:17.083420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:17.083420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:17.083420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:17.083420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:17.083420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:17.083420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:17.083420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:17.083420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:17.083420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:17.083420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:17.084034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 74/144] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:17.088953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77399d0-fd15-4567-9fc2-8b1778c9d58b None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:17.227964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:17.232779 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77399d0-fd15-4567-9fc2-8b1778c9d58b None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:17.282510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6ba25-343d-4780-ad42-1a9e35ac7c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:17.283093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6ba25-343d-4780-ad42-1a9e35ac7c26 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:17.283378 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6ba25-343d-4780-ad42-1a9e35ac7c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:17.283655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6ba25-343d-4780-ad42-1a9e35ac7c26 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:17.284420 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6ba25-343d-4780-ad42-1a9e35ac7c26 None admin] RBAC: Adding query filter params (name=member) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:17.284733 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6ba25-343d-4780-ad42-1a9e35ac7c26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:17.286133 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6ba25-343d-4780-ad42-1a9e35ac7c26 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:17.298068 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 71/145] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:17.304000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7d3a5-54f4-401b-ab3f-d63dec04e983 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:17.356125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39697622-a56d-4318-8081-36473652f22f None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:17.356458 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39697622-a56d-4318-8081-36473652f22f None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:17.356980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39697622-a56d-4318-8081-36473652f22f None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:17.358367 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 72/146] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:26:17.367073 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f1e05-6ac7-4269-800d-11e92be24d53 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:17.367350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f1e05-6ac7-4269-800d-11e92be24d53 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:17.367639 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f1e05-6ac7-4269-800d-11e92be24d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:17.399766 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-901f1e05-6ac7-4269-800d-11e92be24d53 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:17.406479 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-901f1e05-6ac7-4269-800d-11e92be24d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:17.457620 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7d3a5-54f4-401b-ab3f-d63dec04e983 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:17.524498 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6eb1e8-359e-4c15-a726-b90b16654fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:17.525903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6eb1e8-359e-4c15-a726-b90b16654fde None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:17.527195 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6eb1e8-359e-4c15-a726-b90b16654fde None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:17.528232 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6eb1e8-359e-4c15-a726-b90b16654fde None admin] PATH_INFO: `/v3/users/69a2de33463a462ca1305bf7bea04a37` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:17.531556 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6eb1e8-359e-4c15-a726-b90b16654fde None admin] RBAC: Authorizing `identity:get_user(user_id=69a2de33463a462ca1305bf7bea04a37)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:17.533815 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6eb1e8-359e-4c15-a726-b90b16654fde None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:17.535309 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 73/147] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:17.537490 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 75/148] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:17 2022] GET /identity/v3/users/69a2de33463a462ca1305bf7bea04a37 => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:17.543029 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f1e05-6ac7-4269-800d-11e92be24d53 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:17.605525 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6eb1e8-359e-4c15-a726-b90b16654fde None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:17.660313 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:17.693105 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:17.709062 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f1e05-6ac7-4269-800d-11e92be24d53 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:17.755088 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3962c6f-6ae7-4f89-b93e-39bce68b34b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:17.755753 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3962c6f-6ae7-4f89-b93e-39bce68b34b8 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:17.756035 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3962c6f-6ae7-4f89-b93e-39bce68b34b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:17.756311 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3962c6f-6ae7-4f89-b93e-39bce68b34b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:17.797334 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6eb1e8-359e-4c15-a726-b90b16654fde None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:17.857584 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25aba7-ffa4-41dc-a950-bed68fb01b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:17.858422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25aba7-ffa4-41dc-a950-bed68fb01b24 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:17.860716 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25aba7-ffa4-41dc-a950-bed68fb01b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:17.860999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25aba7-ffa4-41dc-a950-bed68fb01b24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:17.875026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25aba7-ffa4-41dc-a950-bed68fb01b24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:17.885976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-ac25aba7-ffa4-41dc-a950-bed68fb01b24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:26:17.885976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:17.885976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:17.885976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:17.885976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:17.885976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:17.885976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:17.885976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:17.885976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:17.885976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:17.885976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:17.885976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:17.885976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:17.885976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:17.885976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:17.885976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:17.886916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:17.886916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:17.886916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:17.886916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:17.886916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:17.886916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:17.886916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:17.886916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:17.886916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:17.886916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:17.886916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:17.886916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:17.886916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:17.886916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:17.886916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:17.886916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:17.887693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:17.887693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:17.887693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:17.887693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:17.887693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:17.887693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:17.887693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:17.887693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:17.887693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:17.887693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:17.887693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:26:17.887693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:17.887693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 76/149] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:26:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:17.895923 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25aba7-ffa4-41dc-a950-bed68fb01b24 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:17.902370 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3962c6f-6ae7-4f89-b93e-39bce68b34b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:17.903959 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3962c6f-6ae7-4f89-b93e-39bce68b34b8 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:18.062610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:18.083381 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:18.087884 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25aba7-ffa4-41dc-a950-bed68fb01b24 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:18.102929 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 74/150] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:26:18.111054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3962c6f-6ae7-4f89-b93e-39bce68b34b8 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:18.151692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423056b-6bef-4504-a4c9-a1f8eee34b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:18.152727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423056b-6bef-4504-a4c9-a1f8eee34b18 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:18.153213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423056b-6bef-4504-a4c9-a1f8eee34b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:18.154275 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423056b-6bef-4504-a4c9-a1f8eee34b18 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:18.155373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c423056b-6bef-4504-a4c9-a1f8eee34b18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:18.155895 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c423056b-6bef-4504-a4c9-a1f8eee34b18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:18.157669 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c423056b-6bef-4504-a4c9-a1f8eee34b18 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:18.171000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 77/151] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:18.176567 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423056b-6bef-4504-a4c9-a1f8eee34b18 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:18.226409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3962c6f-6ae7-4f89-b93e-39bce68b34b8 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:18.268258 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aea3d5-995d-4c67-9f5f-ccad8525eaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:18.269132 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aea3d5-995d-4c67-9f5f-ccad8525eaa4 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:18.269436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aea3d5-995d-4c67-9f5f-ccad8525eaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:18.270898 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aea3d5-995d-4c67-9f5f-ccad8525eaa4 None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:18.275303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aea3d5-995d-4c67-9f5f-ccad8525eaa4 None admin] RBAC: Authorizing `identity:get_project(project_id=d1f168a334d74b6bb18fa28c3f9774d1)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:18.276666 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aea3d5-995d-4c67-9f5f-ccad8525eaa4 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:18.279026 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 75/152] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:18 2022] GET /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:18.306209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aea3d5-995d-4c67-9f5f-ccad8525eaa4 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:18.325158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423056b-6bef-4504-a4c9-a1f8eee34b18 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:18.372633 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62010a6-4d41-4312-a2ed-e75043661f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:18.373868 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62010a6-4d41-4312-a2ed-e75043661f3f None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:18.374368 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62010a6-4d41-4312-a2ed-e75043661f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:18.374866 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62010a6-4d41-4312-a2ed-e75043661f3f None admin] PATH_INFO: `/v3/users/3d0ddf436e7443679ab397306fca9e90` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:18.376313 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62010a6-4d41-4312-a2ed-e75043661f3f None admin] RBAC: Authorizing `identity:get_user(user_id=3d0ddf436e7443679ab397306fca9e90)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:18.377978 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62010a6-4d41-4312-a2ed-e75043661f3f None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:18.378539 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:18.380088 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 78/153] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:18 2022] GET /identity/v3/users/3d0ddf436e7443679ab397306fca9e90 => generated 247 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:18.386302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62010a6-4d41-4312-a2ed-e75043661f3f None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:18.434973 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:18.485961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aea3d5-995d-4c67-9f5f-ccad8525eaa4 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:18.588651 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077e49f-3267-47f5-a2ed-be49381b502f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:18.589294 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077e49f-3267-47f5-a2ed-be49381b502f None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:18.589582 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077e49f-3267-47f5-a2ed-be49381b502f None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:18.589835 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077e49f-3267-47f5-a2ed-be49381b502f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:18.590712 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077e49f-3267-47f5-a2ed-be49381b502f None admin] RBAC: Adding query filter params (role.id=b45aa1c36d974fe79c809afbfcd01945, scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1, user.id=69a2de33463a462ca1305bf7bea04a37) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:18.591590 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077e49f-3267-47f5-a2ed-be49381b502f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:18.593087 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077e49f-3267-47f5-a2ed-be49381b502f None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:18.600271 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62010a6-4d41-4312-a2ed-e75043661f3f None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:18.618422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 76/154] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:26:18 2022] GET /identity/v3/role_assignments?user.id=69a2de33463a462ca1305bf7bea04a37&scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1&role.id=b45aa1c36d974fe79c809afbfcd01945 => generated 608 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:18.645869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637dbf4-f96a-490a-acea-535c640e17b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:18.646961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637dbf4-f96a-490a-acea-535c640e17b2 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:18.647449 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637dbf4-f96a-490a-acea-535c640e17b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:18.647931 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637dbf4-f96a-490a-acea-535c640e17b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:18.723272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9637dbf4-f96a-490a-acea-535c640e17b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:18.725100 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9637dbf4-f96a-490a-acea-535c640e17b2 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:18.755993 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:18.860704 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 79/155] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 18:26:18.865867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077e49f-3267-47f5-a2ed-be49381b502f None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:19.035208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077e49f-3267-47f5-a2ed-be49381b502f None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:19.066099 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:19.078004 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447fb7f-0235-44f7-91eb-fc15d44626ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:19.078920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447fb7f-0235-44f7-91eb-fc15d44626ee None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:19.079197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447fb7f-0235-44f7-91eb-fc15d44626ee None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:19.079473 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447fb7f-0235-44f7-91eb-fc15d44626ee None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:19.083095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447fb7f-0235-44f7-91eb-fc15d44626ee None admin] RBAC: Authorizing `identity:get_project(project_id=d1f168a334d74b6bb18fa28c3f9774d1)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:19.086200 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447fb7f-0235-44f7-91eb-fc15d44626ee None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:19.086880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 77/156] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:18 2022] GET /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:19.092932 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637dbf4-f96a-490a-acea-535c640e17b2 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:19.196104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fc1d4-a4da-4b20-92c8-e897491f66ad None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:19.196393 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fc1d4-a4da-4b20-92c8-e897491f66ad None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:19.196598 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fc1d4-a4da-4b20-92c8-e897491f66ad None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:19.197987 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 78/157] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:26:19.203567 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ed03f-de4e-47d3-af64-6ea261dcf743 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:19.203858 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ed03f-de4e-47d3-af64-6ea261dcf743 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:19.204139 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ed03f-de4e-47d3-af64-6ea261dcf743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:19.225022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637dbf4-f96a-490a-acea-535c640e17b2 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:19.243422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-783ed03f-de4e-47d3-af64-6ea261dcf743 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:19.244649 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-783ed03f-de4e-47d3-af64-6ea261dcf743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:19.260566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:19.287252 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe9a71a-b702-4954-827d-b30dc0a1f3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:19.288085 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9a71a-b702-4954-827d-b30dc0a1f3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:19.288374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9a71a-b702-4954-827d-b30dc0a1f3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:19.288653 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9a71a-b702-4954-827d-b30dc0a1f3c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:19.289500 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe9a71a-b702-4954-827d-b30dc0a1f3c9 None admin] RBAC: Adding query filter params (role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f, scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1, user.id=3d0ddf436e7443679ab397306fca9e90) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:19.289813 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe9a71a-b702-4954-827d-b30dc0a1f3c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:19.291769 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe9a71a-b702-4954-827d-b30dc0a1f3c9 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:19.306513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 80/158] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:26:19 2022] GET /identity/v3/role_assignments?user.id=3d0ddf436e7443679ab397306fca9e90&scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1&role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:19.380088 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 79/159] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:26:19.415517 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe9a71a-b702-4954-827d-b30dc0a1f3c9 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:19.557140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe9a71a-b702-4954-827d-b30dc0a1f3c9 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:19.600685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e4a61-7285-4c6e-b021-8db6c729ff8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:19.602354 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e4a61-7285-4c6e-b021-8db6c729ff8d None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:19.602907 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e4a61-7285-4c6e-b021-8db6c729ff8d None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:19.603388 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e4a61-7285-4c6e-b021-8db6c729ff8d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:19.616002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e4a61-7285-4c6e-b021-8db6c729ff8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:19.625191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-3e6e4a61-7285-4c6e-b021-8db6c729ff8d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:19.625191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:19.625191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:19.625191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:19.625191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:19.625191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:19.625191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:19.625191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:19.625191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:19.625191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:19.625191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:19.625191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:19.625191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:19.625191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:19.625191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:19.625191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:19.626119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:19.626119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:19.626119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:19.626119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:19.626119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:19.626119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:19.626119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:19.626119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:19.626119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:19.626119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:19.626119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:19.626119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:19.626119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:19.626119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:19.626119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:19.626119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:19.626983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:19.626983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:19.626983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:19.626983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:19.626983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:19.626983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:19.626983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:19.626983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:19.626983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:19.626983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:19.626983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:19.626983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:19.629406 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 81/160] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Jun 27 18:26:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:19.635355 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ed03f-de4e-47d3-af64-6ea261dcf743 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:19.721727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e453852-bc82-4e60-b1c9-16403f9054e2 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:19.722355 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e453852-bc82-4e60-b1c9-16403f9054e2 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:19.722864 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e453852-bc82-4e60-b1c9-16403f9054e2 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:19.724455 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 82/161] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:19.730544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105547c-853b-4b66-9987-67799546bd6a None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:19.731140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105547c-853b-4b66-9987-67799546bd6a None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:19.732159 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105547c-853b-4b66-9987-67799546bd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:19.759839 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-d105547c-853b-4b66-9987-67799546bd6a None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:19.761063 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-d105547c-853b-4b66-9987-67799546bd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:19.766934 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ed03f-de4e-47d3-af64-6ea261dcf743 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:19.789748 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:19.806646 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d9491-5b46-490f-89a8-65abb98fbafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:19.812608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d9491-5b46-490f-89a8-65abb98fbafa None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:19.812608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d9491-5b46-490f-89a8-65abb98fbafa None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:19.812608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d9491-5b46-490f-89a8-65abb98fbafa None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:19.812608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d9491-5b46-490f-89a8-65abb98fbafa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:19.812608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d9491-5b46-490f-89a8-65abb98fbafa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:19.812608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d9491-5b46-490f-89a8-65abb98fbafa None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:19.822272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 80/162] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:26:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:19.828790 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d9491-5b46-490f-89a8-65abb98fbafa None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:19.852843 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:19.911617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 83/163] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:26:19.941716 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105547c-853b-4b66-9987-67799546bd6a None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:19.999066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d9491-5b46-490f-89a8-65abb98fbafa None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:20.062220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105547c-853b-4b66-9987-67799546bd6a None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:20.071124 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affaa41-9621-45ec-bc9f-bef7d6f44e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:20.072296 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affaa41-9621-45ec-bc9f-bef7d6f44e15 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:20.072786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affaa41-9621-45ec-bc9f-bef7d6f44e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:20.073272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affaa41-9621-45ec-bc9f-bef7d6f44e15 None admin] PATH_INFO: `/v3/users/69a2de33463a462ca1305bf7bea04a37` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:20.074726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1affaa41-9621-45ec-bc9f-bef7d6f44e15 None admin] RBAC: Authorizing `identity:get_user(user_id=69a2de33463a462ca1305bf7bea04a37)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:20.076936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1affaa41-9621-45ec-bc9f-bef7d6f44e15 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:20.078818 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 81/164] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:19 2022] GET /identity/v3/users/69a2de33463a462ca1305bf7bea04a37 => generated 275 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:20.084325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affaa41-9621-45ec-bc9f-bef7d6f44e15 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:20.100769 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed2a82-40f2-4fe3-badb-c6b87c82e43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:20.102348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed2a82-40f2-4fe3-badb-c6b87c82e43a None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:20.102896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed2a82-40f2-4fe3-badb-c6b87c82e43a None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:20.103387 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed2a82-40f2-4fe3-badb-c6b87c82e43a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:20.118718 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed2a82-40f2-4fe3-badb-c6b87c82e43a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:20.127829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-11ed2a82-40f2-4fe3-badb-c6b87c82e43a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 18:26:20.127829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:20.127829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:20.127829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:20.127829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:20.127829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:20.127829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:20.127829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:20.127829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:20.127829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:20.127829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:20.127829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:20.127829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:20.127829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:20.127829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:20.127829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:20.128768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:20.128768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:20.128768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:20.128768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:20.128768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:20.128768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:20.128768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:20.128768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:20.128768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:20.128768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:20.128768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:20.128768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:20.128768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:20.128768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:20.128768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:20.128768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:20.129527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:20.129527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:20.129527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:20.129527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:20.129527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:20.129527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:20.129527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:20.129527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:20.129527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:20.129527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:20.129527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 18:26:20.129527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:20.137595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 84/165] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:26:20.139120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed2a82-40f2-4fe3-badb-c6b87c82e43a None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:20.173778 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:20.196479 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:20.283231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affaa41-9621-45ec-bc9f-bef7d6f44e15 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:20.340171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed2a82-40f2-4fe3-badb-c6b87c82e43a None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:20.356317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d47fad-956e-4b4c-88b9-88a30fb1895b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:20.358321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d47fad-956e-4b4c-88b9-88a30fb1895b None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:20.359133 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d47fad-956e-4b4c-88b9-88a30fb1895b None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:20.360468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d47fad-956e-4b4c-88b9-88a30fb1895b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:20.411555 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280fac3-9934-4210-9294-3805bf702c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:20.412234 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280fac3-9934-4210-9294-3805bf702c76 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:20.412518 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280fac3-9934-4210-9294-3805bf702c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:20.412804 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280fac3-9934-4210-9294-3805bf702c76 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:20.413696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c280fac3-9934-4210-9294-3805bf702c76 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:20.413963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c280fac3-9934-4210-9294-3805bf702c76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:20.415500 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c280fac3-9934-4210-9294-3805bf702c76 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:20.440397 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 85/166] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:20.441837 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043adce-6347-4335-8764-6303cb3a18ca None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:20.442130 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043adce-6347-4335-8764-6303cb3a18ca None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:20.446488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043adce-6347-4335-8764-6303cb3a18ca None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:20.446488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 86/167] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:20.448767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f43989-4acb-4340-91bf-6fe9a25c8964 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:20.449072 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f43989-4acb-4340-91bf-6fe9a25c8964 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:20.449337 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f43989-4acb-4340-91bf-6fe9a25c8964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:20.462286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d47fad-956e-4b4c-88b9-88a30fb1895b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:20.464214 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d47fad-956e-4b4c-88b9-88a30fb1895b None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:20.480617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-99f43989-4acb-4340-91bf-6fe9a25c8964 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:20.482741 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-99f43989-4acb-4340-91bf-6fe9a25c8964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:20.530778 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:20.572446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 82/168] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 18:26:20.573592 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d47fad-956e-4b4c-88b9-88a30fb1895b None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:20.640176 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 87/169] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:20.640943 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f43989-4acb-4340-91bf-6fe9a25c8964 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:20.682303 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:20.683600 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d47fad-956e-4b4c-88b9-88a30fb1895b None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:20.721240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e212744-ede1-4180-a897-3f5836cc28f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:20.722467 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e212744-ede1-4180-a897-3f5836cc28f2 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:20.722963 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e212744-ede1-4180-a897-3f5836cc28f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:20.723430 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e212744-ede1-4180-a897-3f5836cc28f2 None admin] PATH_INFO: `/v3/users/e50cef24722a4615ad1077c26e19c162` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:20.724872 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e212744-ede1-4180-a897-3f5836cc28f2 None admin] RBAC: Authorizing `identity:get_user(user_id=e50cef24722a4615ad1077c26e19c162)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:20.726546 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e212744-ede1-4180-a897-3f5836cc28f2 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:20.728381 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 83/170] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:20 2022] GET /identity/v3/users/e50cef24722a4615ad1077c26e19c162 => generated 289 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:20.729320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e212744-ede1-4180-a897-3f5836cc28f2 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:20.766759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:20.846638 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e212744-ede1-4180-a897-3f5836cc28f2 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:20.860351 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f43989-4acb-4340-91bf-6fe9a25c8964 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:20.885168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7baad-04a3-4a94-8a1a-a59f3b8fa91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:20.886118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7baad-04a3-4a94-8a1a-a59f3b8fa91d None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:20.886523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7baad-04a3-4a94-8a1a-a59f3b8fa91d None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:20.886895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7baad-04a3-4a94-8a1a-a59f3b8fa91d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:20.898783 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d7baad-04a3-4a94-8a1a-a59f3b8fa91d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:20.910761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-08d7baad-04a3-4a94-8a1a-a59f3b8fa91d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:20.910761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:20.910761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:20.910761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:20.910761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:20.910761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:20.910761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:20.910761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:20.910761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:20.910761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:20.910761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:20.910761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:20.910761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:20.910761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:20.910761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:20.910761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:20.911775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:20.911775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:20.911775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:20.911775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:20.911775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:20.911775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:20.911775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:20.911775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:20.911775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:20.911775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:20.911775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:20.911775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:20.911775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:20.911775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:20.911775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:20.911775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:20.912553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:20.912553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:20.912553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:20.912553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:20.912553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:20.912553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:20.912553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:20.912553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:20.912553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:20.912553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:20.912553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:20.912553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:26:20.912553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 84/171] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:20.915552 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7baad-04a3-4a94-8a1a-a59f3b8fa91d None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:20.933687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87348b1e-ff11-4bb3-ac53-1280a1abc413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:20.935191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87348b1e-ff11-4bb3-ac53-1280a1abc413 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:20.936140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87348b1e-ff11-4bb3-ac53-1280a1abc413 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:20.936440 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87348b1e-ff11-4bb3-ac53-1280a1abc413 None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:20.939112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87348b1e-ff11-4bb3-ac53-1280a1abc413 None admin] RBAC: Authorizing `identity:get_project(project_id=d1f168a334d74b6bb18fa28c3f9774d1)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:20.940676 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87348b1e-ff11-4bb3-ac53-1280a1abc413 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:20.942416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 88/172] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:20 2022] GET /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1 => generated 298 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:20.946433 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87348b1e-ff11-4bb3-ac53-1280a1abc413 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:21.060324 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:21.087687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7baad-04a3-4a94-8a1a-a59f3b8fa91d None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:21.093603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:21.103168 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87348b1e-ff11-4bb3-ac53-1280a1abc413 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:21.139713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02958831-71b8-4412-afd3-d7f5454bb5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:21.140580 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02958831-71b8-4412-afd3-d7f5454bb5b3 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:21.140906 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02958831-71b8-4412-afd3-d7f5454bb5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:21.141134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02958831-71b8-4412-afd3-d7f5454bb5b3 None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:21.142336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02958831-71b8-4412-afd3-d7f5454bb5b3 None admin] RBAC: Authorizing `identity:get_project(project_id=d1f168a334d74b6bb18fa28c3f9774d1)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:21.143549 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26b15d-5005-4fb6-ba7e-8a37c435a5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:21.143812 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02958831-71b8-4412-afd3-d7f5454bb5b3 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:21.144332 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26b15d-5005-4fb6-ba7e-8a37c435a5ef None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:21.144676 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26b15d-5005-4fb6-ba7e-8a37c435a5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:21.145015 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26b15d-5005-4fb6-ba7e-8a37c435a5ef None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:21.145587 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 85/173] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:20 2022] GET /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:21.146509 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02958831-71b8-4412-afd3-d7f5454bb5b3 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:21.150438 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26b15d-5005-4fb6-ba7e-8a37c435a5ef None admin] RBAC: Adding query filter params (name=member) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:21.150816 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26b15d-5005-4fb6-ba7e-8a37c435a5ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:21.152354 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26b15d-5005-4fb6-ba7e-8a37c435a5ef None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:21.163870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 89/174] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:21.165151 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773b404-aa9e-4fed-8a27-d8ad4402d9a2 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:21.165430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773b404-aa9e-4fed-8a27-d8ad4402d9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:21.165702 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773b404-aa9e-4fed-8a27-d8ad4402d9a2 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:21.167102 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 90/175] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:21.167614 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773b404-aa9e-4fed-8a27-d8ad4402d9a2 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:21.286517 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773b404-aa9e-4fed-8a27-d8ad4402d9a2 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:21.286707 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02958831-71b8-4412-afd3-d7f5454bb5b3 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:21.322578 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813ad70-1b2a-402b-aaca-c8b1501e0200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:21.323455 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813ad70-1b2a-402b-aaca-c8b1501e0200 None admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:21.323630 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ce8ac-a8ab-49d1-8054-e7be4b41e4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:21.323750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813ad70-1b2a-402b-aaca-c8b1501e0200 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:21.323992 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813ad70-1b2a-402b-aaca-c8b1501e0200 None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1/users/e50cef24722a4615ad1077c26e19c162/roles/b042cc08697840f5a9ae4066a6f0205c` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:21.324232 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ce8ac-a8ab-49d1-8054-e7be4b41e4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:21.324517 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ce8ac-a8ab-49d1-8054-e7be4b41e4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:21.324794 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ce8ac-a8ab-49d1-8054-e7be4b41e4a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:21.326146 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ce8ac-a8ab-49d1-8054-e7be4b41e4a2 None admin] RBAC: Adding query filter params (role.id=44d7d6bfe7eb441bbc6a301a5b6784ad, scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1, user.id=69a2de33463a462ca1305bf7bea04a37) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:21.326490 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ce8ac-a8ab-49d1-8054-e7be4b41e4a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:21.326582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813ad70-1b2a-402b-aaca-c8b1501e0200 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1f168a334d74b6bb18fa28c3f9774d1, user_id=e50cef24722a4615ad1077c26e19c162, role_id=b042cc08697840f5a9ae4066a6f0205c)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:21.328035 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813ad70-1b2a-402b-aaca-c8b1501e0200 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:21.328146 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ce8ac-a8ab-49d1-8054-e7be4b41e4a2 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:21.335161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 86/176] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:26:21 2022] GET /identity/v3/role_assignments?user.id=69a2de33463a462ca1305bf7bea04a37&scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1&role.id=44d7d6bfe7eb441bbc6a301a5b6784ad => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:21.335970 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ce8ac-a8ab-49d1-8054-e7be4b41e4a2 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:21.350854 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:21.410361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 91/177] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:26:21 2022] PUT /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1/users/e50cef24722a4615ad1077c26e19c162/roles/b042cc08697840f5a9ae4066a6f0205c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:26:21.410361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2aba27-f1c3-4fd2-91b0-ade4ac1dc51e None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:21.410361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2aba27-f1c3-4fd2-91b0-ade4ac1dc51e None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:21.410361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2aba27-f1c3-4fd2-91b0-ade4ac1dc51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:21.444657 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-ee2aba27-f1c3-4fd2-91b0-ade4ac1dc51e None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:21.446199 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ee2aba27-f1c3-4fd2-91b0-ade4ac1dc51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:21.458860 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:21.486718 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ce8ac-a8ab-49d1-8054-e7be4b41e4a2 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:21.537526 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712ce45-f7c6-4fd3-8bb9-e1359ce3f7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:21.538540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712ce45-f7c6-4fd3-8bb9-e1359ce3f7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:21.538789 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712ce45-f7c6-4fd3-8bb9-e1359ce3f7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:21.539069 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712ce45-f7c6-4fd3-8bb9-e1359ce3f7e4 None admin] PATH_INFO: `/v3/users/69a2de33463a462ca1305bf7bea04a37` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:21.540370 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7712ce45-f7c6-4fd3-8bb9-e1359ce3f7e4 None admin] RBAC: Authorizing `identity:get_user(user_id=69a2de33463a462ca1305bf7bea04a37)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:21.541948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7712ce45-f7c6-4fd3-8bb9-e1359ce3f7e4 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:21.543879 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 87/178] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:21 2022] GET /identity/v3/users/69a2de33463a462ca1305bf7bea04a37 => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:21.548488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 92/179] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:21.549990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76edff-edc6-4423-9069-8dd0c5e89141 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:21.550136 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd5c51-7c3e-4a27-a2a1-a06e9310e4ee None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:21.550293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76edff-edc6-4423-9069-8dd0c5e89141 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:21.550412 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd5c51-7c3e-4a27-a2a1-a06e9310e4ee None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:21.550625 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd5c51-7c3e-4a27-a2a1-a06e9310e4ee None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:21.550715 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76edff-edc6-4423-9069-8dd0c5e89141 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:21.552148 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 93/181] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:21.552248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 88/181] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:21.557564 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0a2ec-3bfa-4ce6-b51d-8271150fd57c None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:21.557858 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0a2ec-3bfa-4ce6-b51d-8271150fd57c None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:21.558140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0a2ec-3bfa-4ce6-b51d-8271150fd57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:21.564032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487835c7-7fce-4b64-8a18-60cb032c475d None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:21.564335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487835c7-7fce-4b64-8a18-60cb032c475d None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:21.564597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487835c7-7fce-4b64-8a18-60cb032c475d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:21.579183 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-a7c0a2ec-3bfa-4ce6-b51d-8271150fd57c None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:21.580389 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-a7c0a2ec-3bfa-4ce6-b51d-8271150fd57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:21.589634 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-487835c7-7fce-4b64-8a18-60cb032c475d None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:21.591517 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-487835c7-7fce-4b64-8a18-60cb032c475d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:21.671553 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 94/182] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:21.672252 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0a2ec-3bfa-4ce6-b51d-8271150fd57c None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:21.675267 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 89/183] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:21.676302 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487835c7-7fce-4b64-8a18-60cb032c475d None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:21.686786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:21.765861 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:21.791035 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0a2ec-3bfa-4ce6-b51d-8271150fd57c None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:21.795841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487835c7-7fce-4b64-8a18-60cb032c475d None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:21.829419 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be90fc-8928-4774-8e8d-555e57b3ef12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:21.830063 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be90fc-8928-4774-8e8d-555e57b3ef12 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:21.830343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be90fc-8928-4774-8e8d-555e57b3ef12 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:21.830651 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be90fc-8928-4774-8e8d-555e57b3ef12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:21.831874 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bcd51-ae7b-4833-aaba-5c048ab72a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:21.832652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bcd51-ae7b-4833-aaba-5c048ab72a3b None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:21.832932 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bcd51-ae7b-4833-aaba-5c048ab72a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:21.833242 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bcd51-ae7b-4833-aaba-5c048ab72a3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:21.844509 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872bcd51-ae7b-4833-aaba-5c048ab72a3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:21.852076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-872bcd51-ae7b-4833-aaba-5c048ab72a3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:26:21.852076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:21.852076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:21.852076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:21.852076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:21.852076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:21.852076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:21.852076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:21.852076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:21.852076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:21.852076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:21.852076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:21.852076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:21.852076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:21.852076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:21.852076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:21.852944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:21.852944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:21.852944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:21.852944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:21.852944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:21.852944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:21.852944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:21.852944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:21.852944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:21.852944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:21.852944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:21.852944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:21.852944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:21.852944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:21.852944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:21.852944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:21.853727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:21.853727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:21.853727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:21.853727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:21.853727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:21.853727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:21.853727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:21.853727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:21.853727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:21.853727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:21.853727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:26:21.853727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:26:21.853727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 90/184] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:26:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:21.854365 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bcd51-ae7b-4833-aaba-5c048ab72a3b None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:21.909943 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be90fc-8928-4774-8e8d-555e57b3ef12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:21.911453 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be90fc-8928-4774-8e8d-555e57b3ef12 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:21.932693 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:21.970432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bcd51-ae7b-4833-aaba-5c048ab72a3b None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:21.992186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:22.005997 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95763f42-b2a9-45b5-8ef8-1136656af4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:22.006894 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95763f42-b2a9-45b5-8ef8-1136656af4ef None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:22.007176 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95763f42-b2a9-45b5-8ef8-1136656af4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:22.007449 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95763f42-b2a9-45b5-8ef8-1136656af4ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:22.008612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95763f42-b2a9-45b5-8ef8-1136656af4ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:22.010052 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95763f42-b2a9-45b5-8ef8-1136656af4ef None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:22.011904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 91/185] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:26:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:22.012434 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95763f42-b2a9-45b5-8ef8-1136656af4ef None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:22.027462 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 95/186] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:26:22.028141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be90fc-8928-4774-8e8d-555e57b3ef12 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:22.127173 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95763f42-b2a9-45b5-8ef8-1136656af4ef None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:22.133751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be90fc-8928-4774-8e8d-555e57b3ef12 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:22.163252 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8668275-47f5-46fa-ba0b-74964714eda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:22.164045 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8668275-47f5-46fa-ba0b-74964714eda6 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:22.164327 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8668275-47f5-46fa-ba0b-74964714eda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:22.164610 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8668275-47f5-46fa-ba0b-74964714eda6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:22.167549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:22.170130 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9265c-fd5c-4f76-b3a6-00016a9dec83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:22.170834 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9265c-fd5c-4f76-b3a6-00016a9dec83 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:22.171113 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9265c-fd5c-4f76-b3a6-00016a9dec83 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:22.171384 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9265c-fd5c-4f76-b3a6-00016a9dec83 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:22.172128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa9265c-fd5c-4f76-b3a6-00016a9dec83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:22.178146 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa9265c-fd5c-4f76-b3a6-00016a9dec83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:22.178146 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa9265c-fd5c-4f76-b3a6-00016a9dec83 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:22.178329 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8668275-47f5-46fa-ba0b-74964714eda6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:22.182966 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 96/187] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:22.185222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9265c-fd5c-4f76-b3a6-00016a9dec83 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:22.185755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-c8668275-47f5-46fa-ba0b-74964714eda6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:22.185755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:22.185755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:22.185755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:22.185755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:22.185755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:22.185755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:22.185755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:22.185755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:22.185755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:22.185755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:22.185755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:22.185755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:22.185755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:22.185755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:22.185755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:22.186677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:22.186677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:22.186677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:22.186677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:22.186677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:22.186677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:22.186677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:22.186677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:22.186677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:22.186677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:22.186677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:22.186677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:22.186677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:22.186677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:22.186677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:22.186677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:22.187468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:22.187468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:22.187468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:22.187468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:22.187468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:22.187468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:22.187468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:22.187468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:22.187468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:22.187468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:22.187468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:22.187468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:26:22.187468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 92/188] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Jun 27 18:26:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:22.188436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8668275-47f5-46fa-ba0b-74964714eda6 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:22.230950 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:22.293061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8668275-47f5-46fa-ba0b-74964714eda6 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:22.299825 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9265c-fd5c-4f76-b3a6-00016a9dec83 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:22.328978 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1587c9-683e-4b77-9482-486794be79f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:22.329826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1587c9-683e-4b77-9482-486794be79f6 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:22.330103 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1587c9-683e-4b77-9482-486794be79f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:22.330391 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1587c9-683e-4b77-9482-486794be79f6 None admin] PATH_INFO: `/v3/projects/0ee84742369a4858b3d2514ebfd17159` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:22.331580 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1587c9-683e-4b77-9482-486794be79f6 None admin] RBAC: Authorizing `identity:get_project(project_id=0ee84742369a4858b3d2514ebfd17159)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:22.333039 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1587c9-683e-4b77-9482-486794be79f6 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:22.339631 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 93/189] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:22 2022] GET /identity/v3/projects/0ee84742369a4858b3d2514ebfd17159 => generated 312 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:22.339762 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6179c-ada7-4418-ac21-9449195697f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:22.339867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1587c9-683e-4b77-9482-486794be79f6 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:22.339950 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6179c-ada7-4418-ac21-9449195697f8 None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:22.339950 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6179c-ada7-4418-ac21-9449195697f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:22.339950 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6179c-ada7-4418-ac21-9449195697f8 None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:22.339950 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d6179c-ada7-4418-ac21-9449195697f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:22.339950 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d6179c-ada7-4418-ac21-9449195697f8 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:22.401730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:22.438579 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1587c9-683e-4b77-9482-486794be79f6 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:22.450415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 97/190] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Jun 27 18:26:22 2022] POST /identity/v3/users => generated 297 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:26:22.450415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6179c-ada7-4418-ac21-9449195697f8 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:22.488452 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ab632-6324-4888-955a-ea68cd2d185c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:22.489336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ab632-6324-4888-955a-ea68cd2d185c None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:22.489618 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ab632-6324-4888-955a-ea68cd2d185c None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:22.489895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ab632-6324-4888-955a-ea68cd2d185c None admin] PATH_INFO: `/v3/users/b7cfdc874d574badb083a68b5d1699f8` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:22.512692 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46ab632-6324-4888-955a-ea68cd2d185c None admin] RBAC: Authorizing `identity:get_user(user_id=b7cfdc874d574badb083a68b5d1699f8)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:22.514485 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46ab632-6324-4888-955a-ea68cd2d185c None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:22.517057 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 94/191] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:22 2022] GET /identity/v3/users/b7cfdc874d574badb083a68b5d1699f8 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:22.517699 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ab632-6324-4888-955a-ea68cd2d185c None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:22.564842 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:22.574834 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6179c-ada7-4418-ac21-9449195697f8 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:22.617146 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7fe518-b8fa-484c-8f3b-320442d862c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:22.619237 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fe518-b8fa-484c-8f3b-320442d862c7 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:22.620276 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fe518-b8fa-484c-8f3b-320442d862c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:22.620559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fe518-b8fa-484c-8f3b-320442d862c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:22.621346 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7fe518-b8fa-484c-8f3b-320442d862c7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:22.621659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7fe518-b8fa-484c-8f3b-320442d862c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:22.623711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7fe518-b8fa-484c-8f3b-320442d862c7 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:22.633309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 98/192] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:26:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:22.634193 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7fe518-b8fa-484c-8f3b-320442d862c7 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:22.667455 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ab632-6324-4888-955a-ea68cd2d185c None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:22.695103 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:22.710813 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45cd92-febd-486a-b99c-591f814c168f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:22.711469 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45cd92-febd-486a-b99c-591f814c168f None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:22.711752 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45cd92-febd-486a-b99c-591f814c168f None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:22.712138 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45cd92-febd-486a-b99c-591f814c168f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:22.712983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45cd92-febd-486a-b99c-591f814c168f None admin] RBAC: Adding query filter params (role.id=b45aa1c36d974fe79c809afbfcd01945, scope.project.id=0ee84742369a4858b3d2514ebfd17159, user.id=69a2de33463a462ca1305bf7bea04a37) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:22.713296 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45cd92-febd-486a-b99c-591f814c168f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:22.714782 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45cd92-febd-486a-b99c-591f814c168f None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:22.721293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 95/193] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:26:22 2022] GET /identity/v3/role_assignments?user.id=69a2de33463a462ca1305bf7bea04a37&scope.project.id=0ee84742369a4858b3d2514ebfd17159&role.id=b45aa1c36d974fe79c809afbfcd01945 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:22.722217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45cd92-febd-486a-b99c-591f814c168f None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:22.775649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7fe518-b8fa-484c-8f3b-320442d862c7 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:22.826322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0585f-29b1-485e-b548-196724f0c3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:22.827972 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0585f-29b1-485e-b548-196724f0c3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:22.828481 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0585f-29b1-485e-b548-196724f0c3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:22.829215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0585f-29b1-485e-b548-196724f0c3d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:22.849840 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45cd92-febd-486a-b99c-591f814c168f None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:22.865753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:22.896874 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11843c2b-f4b6-4f61-b643-732fb442dd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:22.897729 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11843c2b-f4b6-4f61-b643-732fb442dd73 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:22.898019 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11843c2b-f4b6-4f61-b643-732fb442dd73 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:22.898296 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11843c2b-f4b6-4f61-b643-732fb442dd73 None admin] PATH_INFO: `/v3/users/b7cfdc874d574badb083a68b5d1699f8` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:22.900079 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11843c2b-f4b6-4f61-b643-732fb442dd73 None admin] RBAC: Authorizing `identity:get_user(user_id=b7cfdc874d574badb083a68b5d1699f8)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:22.901559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11843c2b-f4b6-4f61-b643-732fb442dd73 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:22.908431 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 96/194] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:22 2022] GET /identity/v3/users/b7cfdc874d574badb083a68b5d1699f8 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:22.909394 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11843c2b-f4b6-4f61-b643-732fb442dd73 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:22.930007 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0585f-29b1-485e-b548-196724f0c3d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:22.932549 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0585f-29b1-485e-b548-196724f0c3d8 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:22.989714 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:23.056476 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11843c2b-f4b6-4f61-b643-732fb442dd73 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:23.083288 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 99/195] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:26:23.084713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0585f-29b1-485e-b548-196724f0c3d8 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:23.117179 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dab9ef-b0be-4c63-95c1-04494cf07942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:23.118264 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dab9ef-b0be-4c63-95c1-04494cf07942 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:23.118570 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dab9ef-b0be-4c63-95c1-04494cf07942 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:23.118841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dab9ef-b0be-4c63-95c1-04494cf07942 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:23.160451 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:23.213129 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0585f-29b1-485e-b548-196724f0c3d8 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:23.253961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d93bb2-69c3-4d31-9265-de60fefe5e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:23.255687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d93bb2-69c3-4d31-9265-de60fefe5e0f None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:23.256299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d93bb2-69c3-4d31-9265-de60fefe5e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:23.256775 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d93bb2-69c3-4d31-9265-de60fefe5e0f None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:23.258231 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d93bb2-69c3-4d31-9265-de60fefe5e0f None admin] RBAC: Authorizing `identity:get_project(project_id=4d7f983bf3f84850a4a59357572371d9)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:23.260007 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d93bb2-69c3-4d31-9265-de60fefe5e0f None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:23.261943 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 100/196] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:23 2022] GET /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:23.271467 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d93bb2-69c3-4d31-9265-de60fefe5e0f None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:23.292055 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dab9ef-b0be-4c63-95c1-04494cf07942 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:23.293518 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dab9ef-b0be-4c63-95c1-04494cf07942 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:23.366392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:23.439921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 97/197] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:26:23.444734 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dab9ef-b0be-4c63-95c1-04494cf07942 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:23.478440 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d93bb2-69c3-4d31-9265-de60fefe5e0f None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:23.520415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:23.533843 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76db08-600c-4ad4-ab8b-5720aa8f04b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:23.534571 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76db08-600c-4ad4-ab8b-5720aa8f04b7 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:23.534963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76db08-600c-4ad4-ab8b-5720aa8f04b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:23.535151 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76db08-600c-4ad4-ab8b-5720aa8f04b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:23.535987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76db08-600c-4ad4-ab8b-5720aa8f04b7 None admin] RBAC: Adding query filter params (role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f, scope.project.id=4d7f983bf3f84850a4a59357572371d9, user.id=b7cfdc874d574badb083a68b5d1699f8) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:23.536295 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76db08-600c-4ad4-ab8b-5720aa8f04b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:23.540709 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76db08-600c-4ad4-ab8b-5720aa8f04b7 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:23.553652 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 101/198] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:26:23 2022] GET /identity/v3/role_assignments?user.id=b7cfdc874d574badb083a68b5d1699f8&scope.project.id=4d7f983bf3f84850a4a59357572371d9&role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:23.597258 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167d78b-e3a3-4878-b051-49a471f008c7 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:23.597584 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167d78b-e3a3-4878-b051-49a471f008c7 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:23.597831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167d78b-e3a3-4878-b051-49a471f008c7 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:23.599516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 102/199] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:26:23.604947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4258ce-c19f-4431-9d62-161fc48f2aaa None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:23.605233 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4258ce-c19f-4431-9d62-161fc48f2aaa None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:23.605518 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4258ce-c19f-4431-9d62-161fc48f2aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:23.625222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-ad4258ce-c19f-4431-9d62-161fc48f2aaa None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:23.626487 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ad4258ce-c19f-4431-9d62-161fc48f2aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:23.655071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dab9ef-b0be-4c63-95c1-04494cf07942 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:23.713237 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 103/200] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:26:23.738593 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4258ce-c19f-4431-9d62-161fc48f2aaa None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:23.739535 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b6af2-19b0-46d6-b78c-263fcbb872d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:23.740410 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b6af2-19b0-46d6-b78c-263fcbb872d3 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:23.740689 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b6af2-19b0-46d6-b78c-263fcbb872d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:23.740962 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b6af2-19b0-46d6-b78c-263fcbb872d3 None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:23.746408 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b6af2-19b0-46d6-b78c-263fcbb872d3 None admin] RBAC: Authorizing `identity:get_project(project_id=4d7f983bf3f84850a4a59357572371d9)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:23.749806 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b6af2-19b0-46d6-b78c-263fcbb872d3 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:23.751917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 98/201] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:23 2022] GET /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9 => generated 302 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:23.759530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b6af2-19b0-46d6-b78c-263fcbb872d3 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:23.797486 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:23.878164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b6af2-19b0-46d6-b78c-263fcbb872d3 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:23.879538 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4258ce-c19f-4431-9d62-161fc48f2aaa None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:23.897339 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:23.916340 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b663edb-dd23-4b22-93d1-ecd0be622ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:23.917477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b663edb-dd23-4b22-93d1-ecd0be622ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:23.917955 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b663edb-dd23-4b22-93d1-ecd0be622ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:23.918158 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beddbf8-d1dd-41e2-91fa-97f6776a026a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:23.918678 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b663edb-dd23-4b22-93d1-ecd0be622ef9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:23.919044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beddbf8-d1dd-41e2-91fa-97f6776a026a None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:23.919408 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beddbf8-d1dd-41e2-91fa-97f6776a026a None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:23.919756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beddbf8-d1dd-41e2-91fa-97f6776a026a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:23.920821 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beddbf8-d1dd-41e2-91fa-97f6776a026a None admin] RBAC: Adding query filter params (role.id=44d7d6bfe7eb441bbc6a301a5b6784ad, scope.project.id=4d7f983bf3f84850a4a59357572371d9, user.id=b7cfdc874d574badb083a68b5d1699f8) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:23.921140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beddbf8-d1dd-41e2-91fa-97f6776a026a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:23.922898 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beddbf8-d1dd-41e2-91fa-97f6776a026a None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:23.928926 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 99/202] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:26:23 2022] GET /identity/v3/role_assignments?user.id=b7cfdc874d574badb083a68b5d1699f8&scope.project.id=4d7f983bf3f84850a4a59357572371d9&role.id=44d7d6bfe7eb441bbc6a301a5b6784ad => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:23.931462 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b663edb-dd23-4b22-93d1-ecd0be622ef9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:23.940141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-7b663edb-dd23-4b22-93d1-ecd0be622ef9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 18:26:23.940141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:23.940141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:23.940141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:23.940141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:23.940141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:23.940141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:23.940141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:23.940141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:23.940141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:23.940141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:23.940141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:23.940141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:23.940141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:23.940141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:23.940141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:23.941105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:23.941105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:23.941105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:23.941105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:23.941105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:23.941105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:23.941105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:23.941105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:23.941105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:23.941105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:23.941105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:23.941105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:23.941105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:23.941105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:23.941105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:23.941105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:23.941886 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:23.941886 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:23.941886 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:23.941886 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:23.941886 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:23.941886 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:23.941886 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:23.941886 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:23.941886 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:23.941886 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:23.941886 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 18:26:23.941886 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:23.944877 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 104/203] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:26:23.950719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beddbf8-d1dd-41e2-91fa-97f6776a026a None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:24.059613 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:24.077768 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beddbf8-d1dd-41e2-91fa-97f6776a026a None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:24.125894 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1169df75-b18b-4242-9f06-7819d77e9dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:24.126742 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169df75-b18b-4242-9f06-7819d77e9dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:24.127025 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169df75-b18b-4242-9f06-7819d77e9dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:24.127312 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1169df75-b18b-4242-9f06-7819d77e9dc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:24.128065 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1169df75-b18b-4242-9f06-7819d77e9dc7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:24.128445 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1169df75-b18b-4242-9f06-7819d77e9dc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:24.129836 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1169df75-b18b-4242-9f06-7819d77e9dc7 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:24.144160 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 100/204] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:24.146375 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b663edb-dd23-4b22-93d1-ecd0be622ef9 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:24.286327 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b663edb-dd23-4b22-93d1-ecd0be622ef9 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:24.357967 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4ed31-916d-45f7-a9e9-c98e478ad45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:24.359025 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4ed31-916d-45f7-a9e9-c98e478ad45c None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:24.359311 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4ed31-916d-45f7-a9e9-c98e478ad45c None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:24.359590 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4ed31-916d-45f7-a9e9-c98e478ad45c None admin] PATH_INFO: `/v3/users/e50cef24722a4615ad1077c26e19c162` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:24.361020 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4ed31-916d-45f7-a9e9-c98e478ad45c None admin] RBAC: Authorizing `identity:get_user(user_id=e50cef24722a4615ad1077c26e19c162)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:24.362553 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4ed31-916d-45f7-a9e9-c98e478ad45c None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:24.364379 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 105/205] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:24 2022] GET /identity/v3/users/e50cef24722a4615ad1077c26e19c162 => generated 289 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:24.370123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1169df75-b18b-4242-9f06-7819d77e9dc7 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:24.502907 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1169df75-b18b-4242-9f06-7819d77e9dc7 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:24.549515 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba16b48-192b-46f5-a29b-1b977bbc7e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:24.550179 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba16b48-192b-46f5-a29b-1b977bbc7e86 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:24.550494 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba16b48-192b-46f5-a29b-1b977bbc7e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:24.550777 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba16b48-192b-46f5-a29b-1b977bbc7e86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:24.578056 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:24.639862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba16b48-192b-46f5-a29b-1b977bbc7e86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:24.641241 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba16b48-192b-46f5-a29b-1b977bbc7e86 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:24.769073 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 101/206] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:26:24.774452 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4ed31-916d-45f7-a9e9-c98e478ad45c None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:24.848433 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:24.923035 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4ed31-916d-45f7-a9e9-c98e478ad45c None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:24.980950 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208dbd9-e560-4be5-bc11-559c0ddb41ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:24.982944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208dbd9-e560-4be5-bc11-559c0ddb41ff None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:24.983229 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208dbd9-e560-4be5-bc11-559c0ddb41ff None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:24.983506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208dbd9-e560-4be5-bc11-559c0ddb41ff None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:24.984738 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208dbd9-e560-4be5-bc11-559c0ddb41ff None admin] RBAC: Authorizing `identity:get_project(project_id=d1f168a334d74b6bb18fa28c3f9774d1)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:24.986199 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208dbd9-e560-4be5-bc11-559c0ddb41ff None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:24.989237 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 106/207] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:24 2022] GET /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:24.994097 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba16b48-192b-46f5-a29b-1b977bbc7e86 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:25.076003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:25.146933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba16b48-192b-46f5-a29b-1b977bbc7e86 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:25.177481 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50741d37-3b83-44d4-bc38-18c7f5780b28 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:25.177775 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50741d37-3b83-44d4-bc38-18c7f5780b28 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:25.177957 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50741d37-3b83-44d4-bc38-18c7f5780b28 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:25.179622 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 107/208] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:25.185081 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16e079-5852-4887-9814-cf2aff0f9a66 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:25.185383 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16e079-5852-4887-9814-cf2aff0f9a66 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:25.185655 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16e079-5852-4887-9814-cf2aff0f9a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:25.201075 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e01482-706a-42cb-9a9f-e0352fe133c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:25.202021 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e01482-706a-42cb-9a9f-e0352fe133c1 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:25.202557 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e01482-706a-42cb-9a9f-e0352fe133c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:25.203028 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e01482-706a-42cb-9a9f-e0352fe133c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:25.204088 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e01482-706a-42cb-9a9f-e0352fe133c1 None admin] RBAC: Adding query filter params (role.id=b042cc08697840f5a9ae4066a6f0205c, scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1, user.id=e50cef24722a4615ad1077c26e19c162) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:25.204592 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e01482-706a-42cb-9a9f-e0352fe133c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:25.206225 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e01482-706a-42cb-9a9f-e0352fe133c1 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:25.222514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-1a16e079-5852-4887-9814-cf2aff0f9a66 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:25.224939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-1a16e079-5852-4887-9814-cf2aff0f9a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:25.229026 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 102/209] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:26:24 2022] GET /identity/v3/role_assignments?user.id=e50cef24722a4615ad1077c26e19c162&scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1&role.id=b042cc08697840f5a9ae4066a6f0205c => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:25.307052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 108/210] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:26:25.331388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e01482-706a-42cb-9a9f-e0352fe133c1 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:25.456122 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e01482-706a-42cb-9a9f-e0352fe133c1 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:25.470706 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:25.493702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97aefd6-11cc-4d62-ad6c-ec1a6c6e25a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:25.496637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97aefd6-11cc-4d62-ad6c-ec1a6c6e25a6 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:25.496637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97aefd6-11cc-4d62-ad6c-ec1a6c6e25a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:25.496637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97aefd6-11cc-4d62-ad6c-ec1a6c6e25a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:25.506827 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97aefd6-11cc-4d62-ad6c-ec1a6c6e25a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:25.515620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-d97aefd6-11cc-4d62-ad6c-ec1a6c6e25a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:25.515620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:25.515620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:25.515620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:25.515620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:25.515620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:25.515620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:25.515620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:25.515620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:25.515620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:25.515620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:25.515620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:25.515620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:25.515620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:25.515620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:25.515620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:25.516489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:25.516489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:25.516489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:25.516489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:25.516489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:25.516489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:25.516489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:25.516489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:25.516489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:25.516489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:25.516489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:25.516489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:25.516489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:25.516489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:25.516489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:25.516489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:25.517263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:25.517263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:25.517263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:25.517263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:25.517263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:25.517263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:25.517263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:25.517263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:25.517263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:25.517263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:25.517263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:25.517263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:26:25.519654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 103/211] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:25.525454 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16e079-5852-4887-9814-cf2aff0f9a66 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:25.574673 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:25.638114 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16e079-5852-4887-9814-cf2aff0f9a66 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:25.677199 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050665a-1bc0-4d74-bfb7-e8be83e12bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:25.678219 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050665a-1bc0-4d74-bfb7-e8be83e12bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:25.678720 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050665a-1bc0-4d74-bfb7-e8be83e12bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:25.679211 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050665a-1bc0-4d74-bfb7-e8be83e12bb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:25.680160 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050665a-1bc0-4d74-bfb7-e8be83e12bb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:25.680684 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050665a-1bc0-4d74-bfb7-e8be83e12bb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:25.682255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050665a-1bc0-4d74-bfb7-e8be83e12bb1 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:25.692789 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 109/212] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:25.697845 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97aefd6-11cc-4d62-ad6c-ec1a6c6e25a6 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:25.724682 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bfda4-66bb-4869-957d-4c6ab3c99ad8 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:25.725543 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bfda4-66bb-4869-957d-4c6ab3c99ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:25.726016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bfda4-66bb-4869-957d-4c6ab3c99ad8 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:25.727774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 110/213] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:26:25.739501 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0d309-4efc-484d-83a4-d163b84041ae None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:25.740102 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0d309-4efc-484d-83a4-d163b84041ae None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:25.741067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0d309-4efc-484d-83a4-d163b84041ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:25.770188 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-4fa0d309-4efc-484d-83a4-d163b84041ae None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:25.772275 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-4fa0d309-4efc-484d-83a4-d163b84041ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:25.833581 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97aefd6-11cc-4d62-ad6c-ec1a6c6e25a6 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:25.866517 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 111/214] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:25.877111 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd2e1e-cf48-4dd2-8a80-c12ded7784d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:25.877967 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd2e1e-cf48-4dd2-8a80-c12ded7784d8 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:25.878240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd2e1e-cf48-4dd2-8a80-c12ded7784d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:25.878537 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd2e1e-cf48-4dd2-8a80-c12ded7784d8 None admin] PATH_INFO: `/v3/users/fc8e5aab0e4f496ba69a5b9461948d79` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:25.887226 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0d309-4efc-484d-83a4-d163b84041ae None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:25.893079 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabd2e1e-cf48-4dd2-8a80-c12ded7784d8 None admin] RBAC: Authorizing `identity:get_user(user_id=fc8e5aab0e4f496ba69a5b9461948d79)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:25.894569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabd2e1e-cf48-4dd2-8a80-c12ded7784d8 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:25.896658 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 104/215] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:25 2022] GET /identity/v3/users/fc8e5aab0e4f496ba69a5b9461948d79 => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:25.901728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd2e1e-cf48-4dd2-8a80-c12ded7784d8 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:25.923047 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:25.935478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:25.991022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0d309-4efc-484d-83a4-d163b84041ae None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:26.010905 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd2e1e-cf48-4dd2-8a80-c12ded7784d8 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:26.027067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665aef56-9345-4136-99aa-37a5a51bc3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:26.027927 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665aef56-9345-4136-99aa-37a5a51bc3b4 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:26.028152 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665aef56-9345-4136-99aa-37a5a51bc3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:26.028611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665aef56-9345-4136-99aa-37a5a51bc3b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:26.029842 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665aef56-9345-4136-99aa-37a5a51bc3b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:26.031354 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665aef56-9345-4136-99aa-37a5a51bc3b4 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:26.037433 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 112/216] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:26:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:26.038937 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41602cbe-9d4b-4197-b38f-69252851e97a None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:26.039144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41602cbe-9d4b-4197-b38f-69252851e97a None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:26.039305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41602cbe-9d4b-4197-b38f-69252851e97a None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:26.042425 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 113/217] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:26.042425 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41602cbe-9d4b-4197-b38f-69252851e97a None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:26.054119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d99dd-4b55-46e8-a2ef-39c7b495c96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:26.054791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d99dd-4b55-46e8-a2ef-39c7b495c96a None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:26.055079 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d99dd-4b55-46e8-a2ef-39c7b495c96a None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:26.055357 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d99dd-4b55-46e8-a2ef-39c7b495c96a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:26.133905 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d99dd-4b55-46e8-a2ef-39c7b495c96a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:26.135527 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6d99dd-4b55-46e8-a2ef-39c7b495c96a None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:26.147769 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41602cbe-9d4b-4197-b38f-69252851e97a None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:26.149348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:26.168390 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:26.183551 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4ac87-c87b-44d5-9fb6-7cf5c7e319fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:26.184216 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4ac87-c87b-44d5-9fb6-7cf5c7e319fd None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:26.184989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4ac87-c87b-44d5-9fb6-7cf5c7e319fd None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:26.185272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4ac87-c87b-44d5-9fb6-7cf5c7e319fd None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:26.185980 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4ac87-c87b-44d5-9fb6-7cf5c7e319fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:26.187470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4ac87-c87b-44d5-9fb6-7cf5c7e319fd None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:26.240184 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 105/218] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:26:26.242426 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7235be-218a-421a-91b4-b119db72cc42 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:26.242759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7235be-218a-421a-91b4-b119db72cc42 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:26.242963 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7235be-218a-421a-91b4-b119db72cc42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:26.259453 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-8e7235be-218a-421a-91b4-b119db72cc42 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:26.260669 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-8e7235be-218a-421a-91b4-b119db72cc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:26.274975 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 114/219] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Jun 27 18:26:26 2022] POST /identity/v3/users => generated 297 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:26:26.276026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4ac87-c87b-44d5-9fb6-7cf5c7e319fd None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:26.348718 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 106/220] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:26.379973 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7235be-218a-421a-91b4-b119db72cc42 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:26.415369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4ac87-c87b-44d5-9fb6-7cf5c7e319fd None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:26.462330 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:26.465705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688050c-367d-4972-a7e8-02e86ac99243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:26.466659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688050c-367d-4972-a7e8-02e86ac99243 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:26.466942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688050c-367d-4972-a7e8-02e86ac99243 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:26.467214 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688050c-367d-4972-a7e8-02e86ac99243 None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:26.469181 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c688050c-367d-4972-a7e8-02e86ac99243 None admin] RBAC: Authorizing `identity:get_project(project_id=4d7f983bf3f84850a4a59357572371d9)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:26.470694 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c688050c-367d-4972-a7e8-02e86ac99243 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:26.473092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 115/221] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:26 2022] GET /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:26.479118 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688050c-367d-4972-a7e8-02e86ac99243 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:26.516028 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7235be-218a-421a-91b4-b119db72cc42 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:26.520740 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:26.566128 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f849668-3dcd-4a67-bed3-278f1b3a762b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:26.566969 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f849668-3dcd-4a67-bed3-278f1b3a762b None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:26.567251 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f849668-3dcd-4a67-bed3-278f1b3a762b None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:26.567553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f849668-3dcd-4a67-bed3-278f1b3a762b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:26.583511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f849668-3dcd-4a67-bed3-278f1b3a762b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:26.591549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-5f849668-3dcd-4a67-bed3-278f1b3a762b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:26:26.591549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:26.591549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:26.591549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:26.591549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:26.591549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:26.591549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:26.591549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:26.591549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:26.591549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:26.591549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:26.591549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:26.591549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:26.591549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:26.591549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:26.591549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:26.592492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:26.592492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:26.592492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:26.592492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:26.592492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:26.592492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:26.592492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:26.592492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:26.592492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:26.592492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:26.592492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:26.592492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:26.592492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:26.592492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:26.592492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:26.592492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:26.593287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:26.593287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:26.593287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:26.593287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:26.593287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:26.593287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:26.593287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:26.593287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:26.593287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:26.593287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:26.593287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:26:26.593287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:26:26.593287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 107/222] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:26:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:26:26.600165 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f849668-3dcd-4a67-bed3-278f1b3a762b None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:26.620387 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688050c-367d-4972-a7e8-02e86ac99243 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:26.663543 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62418028-6b0f-486c-9792-81d35e0d81cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:26.664190 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62418028-6b0f-486c-9792-81d35e0d81cd None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:26.664483 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62418028-6b0f-486c-9792-81d35e0d81cd None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:26.664752 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62418028-6b0f-486c-9792-81d35e0d81cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:26.665593 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62418028-6b0f-486c-9792-81d35e0d81cd None admin] RBAC: Adding query filter params (role.id=b45aa1c36d974fe79c809afbfcd01945, scope.project.id=4d7f983bf3f84850a4a59357572371d9, user.id=fc8e5aab0e4f496ba69a5b9461948d79) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:26.665906 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62418028-6b0f-486c-9792-81d35e0d81cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:26.667458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62418028-6b0f-486c-9792-81d35e0d81cd None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:26.674228 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 116/223] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:26:26 2022] GET /identity/v3/role_assignments?user.id=fc8e5aab0e4f496ba69a5b9461948d79&scope.project.id=4d7f983bf3f84850a4a59357572371d9&role.id=b45aa1c36d974fe79c809afbfcd01945 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:26.675640 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2a1e6-c969-45d7-a50e-67a5fc937e11 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:26.675904 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2a1e6-c969-45d7-a50e-67a5fc937e11 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:26.676186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2a1e6-c969-45d7-a50e-67a5fc937e11 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:26.677491 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 117/224] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:26.683478 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49731bf-2ec0-4bbd-a54b-05ca27f069d6 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:26.683825 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49731bf-2ec0-4bbd-a54b-05ca27f069d6 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:26.684018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49731bf-2ec0-4bbd-a54b-05ca27f069d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:26.702893 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-a49731bf-2ec0-4bbd-a54b-05ca27f069d6 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:26.704139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-a49731bf-2ec0-4bbd-a54b-05ca27f069d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:26.730618 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f849668-3dcd-4a67-bed3-278f1b3a762b None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:26.759367 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:26.779162 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a84efd-df0a-4082-84bc-72f7cce55e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:26.779810 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a84efd-df0a-4082-84bc-72f7cce55e67 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:26.780210 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a84efd-df0a-4082-84bc-72f7cce55e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:26.780496 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a84efd-df0a-4082-84bc-72f7cce55e67 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:26.781323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a84efd-df0a-4082-84bc-72f7cce55e67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:26.781640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a84efd-df0a-4082-84bc-72f7cce55e67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:26.783413 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a84efd-df0a-4082-84bc-72f7cce55e67 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:26.793475 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 108/225] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:26.799592 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a84efd-df0a-4082-84bc-72f7cce55e67 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:26.854467 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 118/226] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:26:26.880686 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49731bf-2ec0-4bbd-a54b-05ca27f069d6 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:26.910847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:26.916764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a84efd-df0a-4082-84bc-72f7cce55e67 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:26.960422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36538df9-6c86-4743-8d9f-a8e1a91003cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:26.961952 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36538df9-6c86-4743-8d9f-a8e1a91003cd None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:26.962471 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36538df9-6c86-4743-8d9f-a8e1a91003cd None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:26.963005 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36538df9-6c86-4743-8d9f-a8e1a91003cd None admin] PATH_INFO: `/v3/users/b7cfdc874d574badb083a68b5d1699f8` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:26.964590 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36538df9-6c86-4743-8d9f-a8e1a91003cd None admin] RBAC: Authorizing `identity:get_user(user_id=b7cfdc874d574badb083a68b5d1699f8)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:26.966872 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36538df9-6c86-4743-8d9f-a8e1a91003cd None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:26.969231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 109/227] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:26 2022] GET /identity/v3/users/b7cfdc874d574badb083a68b5d1699f8 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:26.975768 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36538df9-6c86-4743-8d9f-a8e1a91003cd None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:27.035941 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:27.079483 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49731bf-2ec0-4bbd-a54b-05ca27f069d6 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:27.120943 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36538df9-6c86-4743-8d9f-a8e1a91003cd None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:27.156121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1b55b-bad6-42d3-9397-0a1a2b786995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:27.156942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1b55b-bad6-42d3-9397-0a1a2b786995 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:27.157222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1b55b-bad6-42d3-9397-0a1a2b786995 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:27.157494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1b55b-bad6-42d3-9397-0a1a2b786995 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:27.178563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a1b55b-bad6-42d3-9397-0a1a2b786995 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:27.187701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-86a1b55b-bad6-42d3-9397-0a1a2b786995 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:27.187701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:27.187701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:27.187701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:27.187701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:27.187701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:27.187701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:27.187701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:27.187701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:27.187701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:27.187701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:27.187701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:27.187701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:27.187701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:27.187701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:27.187701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:27.188641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:27.188641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:27.188641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:27.188641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:27.188641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:27.188641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:27.188641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:27.188641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:27.188641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:27.188641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:27.188641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:27.188641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:27.188641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:27.188641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:27.188641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:27.188641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:27.189456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:27.189456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:27.189456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:27.189456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:27.189456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:27.189456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:27.189456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:27.189456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:27.189456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:27.189456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:27.189456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:27.189456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:27.193991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 119/228] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Jun 27 18:26:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:27.195163 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1b55b-bad6-42d3-9397-0a1a2b786995 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:27.198429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580212a-315e-4a67-b226-ef5c64dc4811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:27.199617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580212a-315e-4a67-b226-ef5c64dc4811 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:27.200623 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580212a-315e-4a67-b226-ef5c64dc4811 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:27.201117 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580212a-315e-4a67-b226-ef5c64dc4811 None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:27.205208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580212a-315e-4a67-b226-ef5c64dc4811 None admin] RBAC: Authorizing `identity:get_project(project_id=4d7f983bf3f84850a4a59357572371d9)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:27.207089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580212a-315e-4a67-b226-ef5c64dc4811 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:27.209200 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 110/229] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:26 2022] GET /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:27.215609 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580212a-315e-4a67-b226-ef5c64dc4811 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:27.296018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:27.346153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1b55b-bad6-42d3-9397-0a1a2b786995 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:27.377847 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580212a-315e-4a67-b226-ef5c64dc4811 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:27.414995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55f6d4-1c6f-4b3d-a096-a1f45b49e8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:27.416798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55f6d4-1c6f-4b3d-a096-a1f45b49e8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:27.417288 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55f6d4-1c6f-4b3d-a096-a1f45b49e8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:27.417820 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:27.423328 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55f6d4-1c6f-4b3d-a096-a1f45b49e8e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:27.424348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55f6d4-1c6f-4b3d-a096-a1f45b49e8e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:27.424828 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55f6d4-1c6f-4b3d-a096-a1f45b49e8e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:27.426451 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55f6d4-1c6f-4b3d-a096-a1f45b49e8e1 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:27.447460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 120/230] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:26:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:27.452208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15434772-9ecc-4fec-b6d4-2cd98d0b6f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:27.453555 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55f6d4-1c6f-4b3d-a096-a1f45b49e8e1 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:27.454872 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15434772-9ecc-4fec-b6d4-2cd98d0b6f90 None admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:27.457592 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15434772-9ecc-4fec-b6d4-2cd98d0b6f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:27.458962 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15434772-9ecc-4fec-b6d4-2cd98d0b6f90 None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9/users/b7cfdc874d574badb083a68b5d1699f8/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:27.466454 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15434772-9ecc-4fec-b6d4-2cd98d0b6f90 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d7f983bf3f84850a4a59357572371d9, user_id=b7cfdc874d574badb083a68b5d1699f8, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:27.468254 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15434772-9ecc-4fec-b6d4-2cd98d0b6f90 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:27.544131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 111/231] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:26:27 2022] PUT /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9/users/b7cfdc874d574badb083a68b5d1699f8/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:26:27.589061 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55f6d4-1c6f-4b3d-a096-a1f45b49e8e1 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:27.637889 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa5e47-03da-4833-aa15-245819c58a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:27.639332 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa5e47-03da-4833-aa15-245819c58a62 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:27.639836 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa5e47-03da-4833-aa15-245819c58a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:27.640312 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa5e47-03da-4833-aa15-245819c58a62 None admin] PATH_INFO: `/v3/users/b7cfdc874d574badb083a68b5d1699f8` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:27.641911 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa5e47-03da-4833-aa15-245819c58a62 None admin] RBAC: Authorizing `identity:get_user(user_id=b7cfdc874d574badb083a68b5d1699f8)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:27.643705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa5e47-03da-4833-aa15-245819c58a62 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:27.645734 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 121/232] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:27 2022] GET /identity/v3/users/b7cfdc874d574badb083a68b5d1699f8 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:27.651600 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15434772-9ecc-4fec-b6d4-2cd98d0b6f90 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:27.785190 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15434772-9ecc-4fec-b6d4-2cd98d0b6f90 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:27.823493 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6e02d-00f8-48d6-984d-65798c95ab35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:27.824372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6e02d-00f8-48d6-984d-65798c95ab35 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:27.824655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6e02d-00f8-48d6-984d-65798c95ab35 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:27.824928 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6e02d-00f8-48d6-984d-65798c95ab35 None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:27.826722 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6e02d-00f8-48d6-984d-65798c95ab35 None admin] RBAC: Authorizing `identity:get_project(project_id=4d7f983bf3f84850a4a59357572371d9)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:27.828173 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6e02d-00f8-48d6-984d-65798c95ab35 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:27.830247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 112/233] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:27 2022] GET /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:27.835438 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa5e47-03da-4833-aa15-245819c58a62 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:27.858747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:27.984864 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa5e47-03da-4833-aa15-245819c58a62 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:28.032400 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f00f9cc-864c-4a58-a094-a94b9d7718a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:28.033292 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00f9cc-864c-4a58-a094-a94b9d7718a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:28.033576 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00f9cc-864c-4a58-a094-a94b9d7718a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:28.033851 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00f9cc-864c-4a58-a094-a94b9d7718a1 None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9/users/b7cfdc874d574badb083a68b5d1699f8/roles/44d7d6bfe7eb441bbc6a301a5b6784ad` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:28.040193 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f00f9cc-864c-4a58-a094-a94b9d7718a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d7f983bf3f84850a4a59357572371d9, user_id=b7cfdc874d574badb083a68b5d1699f8, role_id=44d7d6bfe7eb441bbc6a301a5b6784ad)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:28.041642 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f00f9cc-864c-4a58-a094-a94b9d7718a1 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:28.145748 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 122/234] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:26:27 2022] PUT /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9/users/b7cfdc874d574badb083a68b5d1699f8/roles/44d7d6bfe7eb441bbc6a301a5b6784ad => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:26:28.916992 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2577a8-3d6d-44dc-b77b-956b24a14fa0 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:28.917500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2577a8-3d6d-44dc-b77b-956b24a14fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:28.917602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2577a8-3d6d-44dc-b77b-956b24a14fa0 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:28.919121 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 113/235] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:28.924439 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64986731-795c-4031-9bca-a7e12cad1466 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:28.924732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64986731-795c-4031-9bca-a7e12cad1466 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:28.925016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64986731-795c-4031-9bca-a7e12cad1466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:28.944303 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-64986731-795c-4031-9bca-a7e12cad1466 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:28.945485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-64986731-795c-4031-9bca-a7e12cad1466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:28.978967 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d7d9f-dfdf-45b4-bce9-b11749b21a1e None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:28.979378 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d7d9f-dfdf-45b4-bce9-b11749b21a1e None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:28.979660 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7d7d9f-dfdf-45b4-bce9-b11749b21a1e None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:28.982714 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 114/236] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:28.988213 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775aed31-5973-4811-91bc-8b358f3f73ad None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:28.988501 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775aed31-5973-4811-91bc-8b358f3f73ad None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:28.989231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775aed31-5973-4811-91bc-8b358f3f73ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:29.012417 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-775aed31-5973-4811-91bc-8b358f3f73ad None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:29.013714 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-775aed31-5973-4811-91bc-8b358f3f73ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:29.036727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:29.044974 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 123/237] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:29.047308 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f825e-8c91-4702-b113-a2a20bafc983 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:29.047908 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f825e-8c91-4702-b113-a2a20bafc983 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:29.048443 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f825e-8c91-4702-b113-a2a20bafc983 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:29.055742 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 124/238] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:29.058059 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795569d-1300-4e94-bf5f-ad8a98a861a0 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:29.058550 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795569d-1300-4e94-bf5f-ad8a98a861a0 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:29.059002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795569d-1300-4e94-bf5f-ad8a98a861a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:29.066771 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:29.083855 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-9795569d-1300-4e94-bf5f-ad8a98a861a0 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:29.085364 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-9795569d-1300-4e94-bf5f-ad8a98a861a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:29.094506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 115/239] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:26:29.095622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775aed31-5973-4811-91bc-8b358f3f73ad None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:29.161702 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 125/240] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:29.163375 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795569d-1300-4e94-bf5f-ad8a98a861a0 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:29.213620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775aed31-5973-4811-91bc-8b358f3f73ad None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:29.260171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596aedcd-e07c-4032-930a-63b94a619f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:29.261280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aedcd-e07c-4032-930a-63b94a619f24 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:29.261721 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aedcd-e07c-4032-930a-63b94a619f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:29.262153 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596aedcd-e07c-4032-930a-63b94a619f24 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:29.274428 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596aedcd-e07c-4032-930a-63b94a619f24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:29.275386 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795569d-1300-4e94-bf5f-ad8a98a861a0 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:29.283540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-596aedcd-e07c-4032-930a-63b94a619f24 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:29.283540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:29.283540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:29.283540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:29.283540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:29.283540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:29.283540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:29.283540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:29.283540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:29.283540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:29.283540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:29.283540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:29.283540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:29.283540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:29.283540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:29.283540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:29.284419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:29.284419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:29.284419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:29.284419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:29.284419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:29.284419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:29.284419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:29.284419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:29.284419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:29.284419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:29.284419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:29.284419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:29.284419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:29.284419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:29.284419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:29.284419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:29.285202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:29.285202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:29.285202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:29.285202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:29.285202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:29.285202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:29.285202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:29.285202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:29.285202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:29.285202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:29.285202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:29.285202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:26:29.287575 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 116/241] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:29.293173 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596aedcd-e07c-4032-930a-63b94a619f24 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:29.311959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:29.320906 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde788cf-441a-476e-b9ac-68f7c0c6536c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:29.322058 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde788cf-441a-476e-b9ac-68f7c0c6536c None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:29.322356 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde788cf-441a-476e-b9ac-68f7c0c6536c None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:29.322662 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde788cf-441a-476e-b9ac-68f7c0c6536c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:29.329229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:29.338273 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde788cf-441a-476e-b9ac-68f7c0c6536c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:29.348944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-fde788cf-441a-476e-b9ac-68f7c0c6536c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 18:26:29.348944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:29.348944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:29.348944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:29.348944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:29.348944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:29.348944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:29.348944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:29.348944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:29.348944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:29.348944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:29.348944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:29.348944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:29.348944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:29.348944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:29.348944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:29.355298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:29.355298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:29.355298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:29.355298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:29.355298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:29.355298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:29.355298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:29.355298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:29.355298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:29.355298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:29.355298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:29.355298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:29.355298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:29.355298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:29.355298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:29.355298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:29.356092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:29.356092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:29.356092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:29.356092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:29.356092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:29.356092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:29.356092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:29.356092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:29.356092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:29.356092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:29.356092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 18:26:29.356092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:29.356092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 126/242] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:29.356092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde788cf-441a-476e-b9ac-68f7c0c6536c None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:29.412266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596aedcd-e07c-4032-930a-63b94a619f24 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:29.450497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f6290-0519-4885-978c-7f36626d2050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:29.452014 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f6290-0519-4885-978c-7f36626d2050 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:29.452468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f6290-0519-4885-978c-7f36626d2050 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:29.452937 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f6290-0519-4885-978c-7f36626d2050 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:29.454274 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f6290-0519-4885-978c-7f36626d2050 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:29.455911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f6290-0519-4885-978c-7f36626d2050 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:29.457697 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 117/243] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:26:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:29.458889 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f6290-0519-4885-978c-7f36626d2050 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:29.483999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde788cf-441a-476e-b9ac-68f7c0c6536c None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:29.521614 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc053f90-65e2-48f6-995f-f7d0f6d0ddb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:29.522675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc053f90-65e2-48f6-995f-f7d0f6d0ddb2 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:29.522783 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc053f90-65e2-48f6-995f-f7d0f6d0ddb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:29.523179 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc053f90-65e2-48f6-995f-f7d0f6d0ddb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:29.524014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc053f90-65e2-48f6-995f-f7d0f6d0ddb2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:29.524370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc053f90-65e2-48f6-995f-f7d0f6d0ddb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:29.526168 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc053f90-65e2-48f6-995f-f7d0f6d0ddb2 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:29.536226 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 127/244] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:29.536742 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc053f90-65e2-48f6-995f-f7d0f6d0ddb2 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:29.568570 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f6290-0519-4885-978c-7f36626d2050 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:29.571347 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:29.596365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:29.605655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70d452-81c3-4690-b173-a2a87a5e52bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:29.608022 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70d452-81c3-4690-b173-a2a87a5e52bd None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:29.608417 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70d452-81c3-4690-b173-a2a87a5e52bd None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:29.608831 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70d452-81c3-4690-b173-a2a87a5e52bd None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:29.609690 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70d452-81c3-4690-b173-a2a87a5e52bd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:29.610095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70d452-81c3-4690-b173-a2a87a5e52bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:29.612083 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70d452-81c3-4690-b173-a2a87a5e52bd None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:29.622518 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 118/245] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:29.623761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70d452-81c3-4690-b173-a2a87a5e52bd None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:29.654617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc053f90-65e2-48f6-995f-f7d0f6d0ddb2 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:29.693250 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a57e0a-a3bf-4f19-9c7b-51bfeeed5c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:29.694289 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a57e0a-a3bf-4f19-9c7b-51bfeeed5c47 None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:29.694742 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a57e0a-a3bf-4f19-9c7b-51bfeeed5c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:29.695231 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a57e0a-a3bf-4f19-9c7b-51bfeeed5c47 None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:29.696225 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a57e0a-a3bf-4f19-9c7b-51bfeeed5c47 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:29.697961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a57e0a-a3bf-4f19-9c7b-51bfeeed5c47 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:29.738829 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70d452-81c3-4690-b173-a2a87a5e52bd None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:29.770065 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 128/246] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Jun 27 18:26:29 2022] POST /identity/v3/users => generated 293 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:26:29.776056 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a57e0a-a3bf-4f19-9c7b-51bfeeed5c47 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:29.782907 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac349e-431f-488a-a835-facdb382d6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:29.783820 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac349e-431f-488a-a835-facdb382d6aa None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:29.784139 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac349e-431f-488a-a835-facdb382d6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:29.784376 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac349e-431f-488a-a835-facdb382d6aa None admin] PATH_INFO: `/v3/users/fc8e5aab0e4f496ba69a5b9461948d79` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:29.785664 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac349e-431f-488a-a835-facdb382d6aa None admin] RBAC: Authorizing `identity:get_user(user_id=fc8e5aab0e4f496ba69a5b9461948d79)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:29.794418 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac349e-431f-488a-a835-facdb382d6aa None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:29.794418 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 119/247] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:29 2022] GET /identity/v3/users/fc8e5aab0e4f496ba69a5b9461948d79 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:29.795470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac349e-431f-488a-a835-facdb382d6aa None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:29.847963 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:29.915190 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a57e0a-a3bf-4f19-9c7b-51bfeeed5c47 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:29.927613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:29.960896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ae739-d357-44ed-9ffe-6b732fdb4222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:29.962272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ae739-d357-44ed-9ffe-6b732fdb4222 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:29.962960 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ae739-d357-44ed-9ffe-6b732fdb4222 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:29.963618 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ae739-d357-44ed-9ffe-6b732fdb4222 None admin] PATH_INFO: `/v3/users/1dc3d318f83a41a0aa4b852a15721b78` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:29.981264 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ae739-d357-44ed-9ffe-6b732fdb4222 None admin] RBAC: Authorizing `identity:get_user(user_id=1dc3d318f83a41a0aa4b852a15721b78)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:29.983474 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ae739-d357-44ed-9ffe-6b732fdb4222 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:29.985482 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 129/248] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:29 2022] GET /identity/v3/users/1dc3d318f83a41a0aa4b852a15721b78 => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:29.991324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ae739-d357-44ed-9ffe-6b732fdb4222 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:30.018065 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac349e-431f-488a-a835-facdb382d6aa None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:30.067721 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b10dbe-dd2f-406c-82a9-21c73ca334b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:30.069358 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b10dbe-dd2f-406c-82a9-21c73ca334b0 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:30.070097 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b10dbe-dd2f-406c-82a9-21c73ca334b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:30.070640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b10dbe-dd2f-406c-82a9-21c73ca334b0 None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:30.072539 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b10dbe-dd2f-406c-82a9-21c73ca334b0 None admin] RBAC: Authorizing `identity:get_project(project_id=4d7f983bf3f84850a4a59357572371d9)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:30.074673 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b10dbe-dd2f-406c-82a9-21c73ca334b0 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:30.077724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 120/249] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:29 2022] GET /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9 => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:30.084188 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b10dbe-dd2f-406c-82a9-21c73ca334b0 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:30.131341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ae739-d357-44ed-9ffe-6b732fdb4222 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:30.180660 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea5625-2d25-4a73-9325-69d6a185a677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:30.181451 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea5625-2d25-4a73-9325-69d6a185a677 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:30.181815 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea5625-2d25-4a73-9325-69d6a185a677 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:30.182000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea5625-2d25-4a73-9325-69d6a185a677 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:30.217496 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b10dbe-dd2f-406c-82a9-21c73ca334b0 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:30.219395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:30.236725 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:30.269529 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04688e32-eb09-45ed-8871-8552ff7a4c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:30.270764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04688e32-eb09-45ed-8871-8552ff7a4c14 None admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:30.271054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04688e32-eb09-45ed-8871-8552ff7a4c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:30.271490 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04688e32-eb09-45ed-8871-8552ff7a4c14 None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9/users/fc8e5aab0e4f496ba69a5b9461948d79/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:30.274228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04688e32-eb09-45ed-8871-8552ff7a4c14 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d7f983bf3f84850a4a59357572371d9, user_id=fc8e5aab0e4f496ba69a5b9461948d79, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:30.275928 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04688e32-eb09-45ed-8871-8552ff7a4c14 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:30.278879 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea5625-2d25-4a73-9325-69d6a185a677 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:30.280846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea5625-2d25-4a73-9325-69d6a185a677 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:30.373460 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 121/250] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:26:30 2022] PUT /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9/users/fc8e5aab0e4f496ba69a5b9461948d79/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:26:30.375249 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4a8b6-6bbf-4e55-8f1c-41b31150775a None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:30.375713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4a8b6-6bbf-4e55-8f1c-41b31150775a None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:30.376161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4a8b6-6bbf-4e55-8f1c-41b31150775a None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:30.379423 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 122/251] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:30.384373 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4681bf-2ac3-446d-8d78-cfec20a10480 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:30.384961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4681bf-2ac3-446d-8d78-cfec20a10480 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:30.385444 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4681bf-2ac3-446d-8d78-cfec20a10480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:30.409348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 130/252] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 18:26:30.414751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea5625-2d25-4a73-9325-69d6a185a677 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:30.421880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-ae4681bf-2ac3-446d-8d78-cfec20a10480 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:30.423506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-ae4681bf-2ac3-446d-8d78-cfec20a10480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:30.556383 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:30.592579 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 123/253] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:30.613192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452eefa-f879-4e29-bcee-9d8658622fc4 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:30.613497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452eefa-f879-4e29-bcee-9d8658622fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:30.613742 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452eefa-f879-4e29-bcee-9d8658622fc4 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:30.615257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 124/254] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:26:30.625435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc4666-4efc-4922-8ac0-a40eb3122560 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:30.625741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc4666-4efc-4922-8ac0-a40eb3122560 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:30.625840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea5625-2d25-4a73-9325-69d6a185a677 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:30.625934 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc4666-4efc-4922-8ac0-a40eb3122560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:30.658920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-32bc4666-4efc-4922-8ac0-a40eb3122560 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:30.660173 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-32bc4666-4efc-4922-8ac0-a40eb3122560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:30.685146 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba147228-5775-4554-bcaf-0e5188f459e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:30.686190 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba147228-5775-4554-bcaf-0e5188f459e5 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:30.686505 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba147228-5775-4554-bcaf-0e5188f459e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:30.686887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba147228-5775-4554-bcaf-0e5188f459e5 None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:30.688121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba147228-5775-4554-bcaf-0e5188f459e5 None admin] RBAC: Authorizing `identity:get_project(project_id=4d7f983bf3f84850a4a59357572371d9)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:30.689697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba147228-5775-4554-bcaf-0e5188f459e5 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:30.692506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 131/255] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:30 2022] GET /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:30.693467 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba147228-5775-4554-bcaf-0e5188f459e5 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:30.740594 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 125/256] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 27 18:26:30.740594 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc4666-4efc-4922-8ac0-a40eb3122560 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:30.805507 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:30.839079 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba147228-5775-4554-bcaf-0e5188f459e5 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:30.904328 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf59bce-a3b3-4ead-8ecc-bfbd192252e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:30.904960 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf59bce-a3b3-4ead-8ecc-bfbd192252e2 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:30.905244 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf59bce-a3b3-4ead-8ecc-bfbd192252e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:30.905519 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf59bce-a3b3-4ead-8ecc-bfbd192252e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:30.914741 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:30.927322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf59bce-a3b3-4ead-8ecc-bfbd192252e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:30.935165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-fbf59bce-a3b3-4ead-8ecc-bfbd192252e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:26:30.935165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:30.935165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:30.935165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:30.935165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:30.935165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:30.935165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:30.935165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:30.935165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:30.935165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:30.935165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:30.935165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:30.935165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:30.935165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:30.935165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:30.935165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:30.936029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:30.936029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:30.936029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:30.936029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:30.936029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:30.936029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:30.936029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:30.936029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:30.936029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:30.936029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:30.936029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:30.936029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:30.936029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:30.936029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:30.936029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:30.936029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:30.936814 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:30.936814 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:30.936814 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:30.936814 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:30.936814 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:30.936814 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:30.936814 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:30.936814 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:30.936814 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:30.936814 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:30.936814 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:26:30.936814 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:30.937890 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 132/257] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:26:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:30.941476 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf59bce-a3b3-4ead-8ecc-bfbd192252e2 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:30.972378 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc4666-4efc-4922-8ac0-a40eb3122560 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:31.041027 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986051e-ea72-4d4a-baea-27e058c66983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:31.041786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986051e-ea72-4d4a-baea-27e058c66983 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:31.041962 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986051e-ea72-4d4a-baea-27e058c66983 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:31.042238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986051e-ea72-4d4a-baea-27e058c66983 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:31.043100 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4986051e-ea72-4d4a-baea-27e058c66983 None admin] RBAC: Adding query filter params (role.id=b042cc08697840f5a9ae4066a6f0205c, scope.project.id=4d7f983bf3f84850a4a59357572371d9, user.id=1dc3d318f83a41a0aa4b852a15721b78) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:31.043413 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4986051e-ea72-4d4a-baea-27e058c66983 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:31.045310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4986051e-ea72-4d4a-baea-27e058c66983 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:31.062955 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 126/258] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:26:30 2022] GET /identity/v3/role_assignments?user.id=1dc3d318f83a41a0aa4b852a15721b78&scope.project.id=4d7f983bf3f84850a4a59357572371d9&role.id=b042cc08697840f5a9ae4066a6f0205c => generated 259 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:31.064829 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986051e-ea72-4d4a-baea-27e058c66983 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:31.201669 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf59bce-a3b3-4ead-8ecc-bfbd192252e2 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:31.304654 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b72a7-b2ae-4dfb-8327-c23581d88e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:31.305493 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b72a7-b2ae-4dfb-8327-c23581d88e2a None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:31.305782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b72a7-b2ae-4dfb-8327-c23581d88e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:31.306058 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b72a7-b2ae-4dfb-8327-c23581d88e2a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:31.311849 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:31.337483 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b72a7-b2ae-4dfb-8327-c23581d88e2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:31.344606 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986051e-ea72-4d4a-baea-27e058c66983 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:31.349162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-588b72a7-b2ae-4dfb-8327-c23581d88e2a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:31.349162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:31.349162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:31.349162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:31.349162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:31.349162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:31.349162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:31.349162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:31.349162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:31.349162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:31.349162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:31.349162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:31.349162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:31.349162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:31.349162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:31.349162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:31.350075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:31.350075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:31.350075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:31.350075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:31.350075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:31.350075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:31.350075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:31.350075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:31.350075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:31.350075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:31.350075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:31.350075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:31.350075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:31.350075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:31.350075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:31.350075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:31.350989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:31.350989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:31.350989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:31.350989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:31.350989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:31.350989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:31.350989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:31.350989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:31.350989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:31.350989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:31.350989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:31.350989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:31.350989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 133/259] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Jun 27 18:26:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:31.357904 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b72a7-b2ae-4dfb-8327-c23581d88e2a None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:31.441972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b84757-0fda-42aa-8ea6-3efaac1a9b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:31.442671 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b84757-0fda-42aa-8ea6-3efaac1a9b33 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:31.442958 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b84757-0fda-42aa-8ea6-3efaac1a9b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:31.443235 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b84757-0fda-42aa-8ea6-3efaac1a9b33 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:31.443994 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b84757-0fda-42aa-8ea6-3efaac1a9b33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:31.444303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b84757-0fda-42aa-8ea6-3efaac1a9b33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:31.448511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b84757-0fda-42aa-8ea6-3efaac1a9b33 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:31.466402 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 127/260] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:31.472684 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b84757-0fda-42aa-8ea6-3efaac1a9b33 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:31.486788 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:31.581370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b72a7-b2ae-4dfb-8327-c23581d88e2a None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:31.664571 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f97e3-8c22-44df-9819-de28568f8410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:31.665870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f97e3-8c22-44df-9819-de28568f8410 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:31.666397 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f97e3-8c22-44df-9819-de28568f8410 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:31.666883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f97e3-8c22-44df-9819-de28568f8410 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:31.667870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f97e3-8c22-44df-9819-de28568f8410 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:31.668371 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f97e3-8c22-44df-9819-de28568f8410 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:31.671137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f97e3-8c22-44df-9819-de28568f8410 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:31.693442 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 134/261] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:26:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:31.695488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f97e3-8c22-44df-9819-de28568f8410 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:31.772580 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b84757-0fda-42aa-8ea6-3efaac1a9b33 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:31.828166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96c478-bb9a-4d8d-99d1-b72c5a756bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:31.828166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96c478-bb9a-4d8d-99d1-b72c5a756bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:31.828166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96c478-bb9a-4d8d-99d1-b72c5a756bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:31.828166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96c478-bb9a-4d8d-99d1-b72c5a756bb9 None admin] PATH_INFO: `/v3/users/b7cfdc874d574badb083a68b5d1699f8` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:31.829198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a96c478-bb9a-4d8d-99d1-b72c5a756bb9 None admin] RBAC: Authorizing `identity:get_user(user_id=b7cfdc874d574badb083a68b5d1699f8)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:31.830700 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a96c478-bb9a-4d8d-99d1-b72c5a756bb9 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:31.833263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 128/262] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:31 2022] GET /identity/v3/users/b7cfdc874d574badb083a68b5d1699f8 => generated 283 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:31.839609 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96c478-bb9a-4d8d-99d1-b72c5a756bb9 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:31.867838 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:31.963542 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f97e3-8c22-44df-9819-de28568f8410 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:31.991632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:32.018583 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96c478-bb9a-4d8d-99d1-b72c5a756bb9 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:32.026727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082bb82-a59f-47db-925f-251c542de711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:32.027869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082bb82-a59f-47db-925f-251c542de711 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:32.028869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082bb82-a59f-47db-925f-251c542de711 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:32.029526 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082bb82-a59f-47db-925f-251c542de711 None admin] PATH_INFO: `/v3/users/b7cfdc874d574badb083a68b5d1699f8` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:32.033955 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9082bb82-a59f-47db-925f-251c542de711 None admin] RBAC: Authorizing `identity:get_user(user_id=b7cfdc874d574badb083a68b5d1699f8)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:32.035697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9082bb82-a59f-47db-925f-251c542de711 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:32.039509 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 135/263] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:31 2022] GET /identity/v3/users/b7cfdc874d574badb083a68b5d1699f8 => generated 283 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:32.042875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082bb82-a59f-47db-925f-251c542de711 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:32.090193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8ddd0-3ebb-461f-b459-7de1f8195600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:32.092400 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8ddd0-3ebb-461f-b459-7de1f8195600 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:32.092910 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8ddd0-3ebb-461f-b459-7de1f8195600 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:32.093376 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8ddd0-3ebb-461f-b459-7de1f8195600 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:32.182655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8ddd0-3ebb-461f-b459-7de1f8195600 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:32.184167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8ddd0-3ebb-461f-b459-7de1f8195600 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:32.201602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:32.252506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082bb82-a59f-47db-925f-251c542de711 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:32.317994 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 129/264] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:26:32.326606 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8ddd0-3ebb-461f-b459-7de1f8195600 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:32.334618 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff501f-b535-4933-baa9-a2c83bf27dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:32.335419 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff501f-b535-4933-baa9-a2c83bf27dca None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:32.342413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff501f-b535-4933-baa9-a2c83bf27dca None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:32.342413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff501f-b535-4933-baa9-a2c83bf27dca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:32.470772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:32.510815 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff501f-b535-4933-baa9-a2c83bf27dca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:32.512282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff501f-b535-4933-baa9-a2c83bf27dca None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:32.537236 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8ddd0-3ebb-461f-b459-7de1f8195600 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:32.580297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:32.625450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312f96b-9dec-4cc6-bf4c-c39398aee57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:32.627454 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312f96b-9dec-4cc6-bf4c-c39398aee57e None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:32.627740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312f96b-9dec-4cc6-bf4c-c39398aee57e None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:32.628013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312f96b-9dec-4cc6-bf4c-c39398aee57e None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:32.629854 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a312f96b-9dec-4cc6-bf4c-c39398aee57e None admin] RBAC: Authorizing `identity:get_project(project_id=4d7f983bf3f84850a4a59357572371d9)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:32.631289 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a312f96b-9dec-4cc6-bf4c-c39398aee57e None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:32.632998 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 130/265] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:32 2022] GET /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9 => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:32.641988 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 136/266] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:26:32.642909 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312f96b-9dec-4cc6-bf4c-c39398aee57e None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:32.647255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff501f-b535-4933-baa9-a2c83bf27dca None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:32.806358 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:32.839233 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff501f-b535-4933-baa9-a2c83bf27dca None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:32.915978 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632f079-801d-4b4b-bf86-29f8af0700c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:32.917178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632f079-801d-4b4b-bf86-29f8af0700c1 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:32.917685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632f079-801d-4b4b-bf86-29f8af0700c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:32.918155 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632f079-801d-4b4b-bf86-29f8af0700c1 None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:32.919689 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632f079-801d-4b4b-bf86-29f8af0700c1 None admin] RBAC: Authorizing `identity:get_project(project_id=4d7f983bf3f84850a4a59357572371d9)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:32.921383 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632f079-801d-4b4b-bf86-29f8af0700c1 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:32.923617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 137/267] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:32 2022] GET /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9 => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:32.931581 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632f079-801d-4b4b-bf86-29f8af0700c1 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:32.932595 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312f96b-9dec-4cc6-bf4c-c39398aee57e None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:33.012534 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a8f71-a0a5-4a1e-9b8a-170325f01b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:33.013574 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a8f71-a0a5-4a1e-9b8a-170325f01b28 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:33.014008 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a8f71-a0a5-4a1e-9b8a-170325f01b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:33.014478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a8f71-a0a5-4a1e-9b8a-170325f01b28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:33.015469 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a8f71-a0a5-4a1e-9b8a-170325f01b28 None admin] RBAC: Adding query filter params (role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f, scope.project.id=4d7f983bf3f84850a4a59357572371d9, user.id=b7cfdc874d574badb083a68b5d1699f8) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:33.015925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a8f71-a0a5-4a1e-9b8a-170325f01b28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:33.017512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a8f71-a0a5-4a1e-9b8a-170325f01b28 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:33.036962 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 131/268] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:26:32 2022] GET /identity/v3/role_assignments?user.id=b7cfdc874d574badb083a68b5d1699f8&scope.project.id=4d7f983bf3f84850a4a59357572371d9&role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 608 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:33.067628 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea2b94-cbf0-4b76-8bd7-51b7dae538c4 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:33.068258 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea2b94-cbf0-4b76-8bd7-51b7dae538c4 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:33.068738 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea2b94-cbf0-4b76-8bd7-51b7dae538c4 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:33.071888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 132/269] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:33.077174 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aaf073-3b93-485a-b2dc-48ec68ef469c None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:33.077766 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aaf073-3b93-485a-b2dc-48ec68ef469c None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:33.078264 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aaf073-3b93-485a-b2dc-48ec68ef469c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:33.119537 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-31aaf073-3b93-485a-b2dc-48ec68ef469c None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:33.121487 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-31aaf073-3b93-485a-b2dc-48ec68ef469c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:33.158596 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632f079-801d-4b4b-bf86-29f8af0700c1 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:33.264072 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfbb78-d765-40b6-97f3-1e4ed75db35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:33.265425 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfbb78-d765-40b6-97f3-1e4ed75db35f None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:33.265859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfbb78-d765-40b6-97f3-1e4ed75db35f None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:33.266352 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfbb78-d765-40b6-97f3-1e4ed75db35f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:33.272002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfbb78-d765-40b6-97f3-1e4ed75db35f None admin] RBAC: Adding query filter params (role.id=44d7d6bfe7eb441bbc6a301a5b6784ad, scope.project.id=4d7f983bf3f84850a4a59357572371d9, user.id=b7cfdc874d574badb083a68b5d1699f8) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:33.272002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfbb78-d765-40b6-97f3-1e4ed75db35f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:33.272002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfbb78-d765-40b6-97f3-1e4ed75db35f None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:33.284370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 138/270] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:26:32 2022] GET /identity/v3/role_assignments?user.id=b7cfdc874d574badb083a68b5d1699f8&scope.project.id=4d7f983bf3f84850a4a59357572371d9&role.id=44d7d6bfe7eb441bbc6a301a5b6784ad => generated 608 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:33.290794 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:33.292206 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d9ed2-c4fc-49b0-b059-bbc827ea0178 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:33.292690 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d9ed2-c4fc-49b0-b059-bbc827ea0178 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:33.293123 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d9ed2-c4fc-49b0-b059-bbc827ea0178 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:33.295259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 139/271] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:33 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:33.304497 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe26d6-9a98-4e15-87d2-27e7d4ad5e30 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:33.305021 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe26d6-9a98-4e15-87d2-27e7d4ad5e30 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:33.311159 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe26d6-9a98-4e15-87d2-27e7d4ad5e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:33.335335 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-8bbe26d6-9a98-4e15-87d2-27e7d4ad5e30 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:33.336562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-8bbe26d6-9a98-4e15-87d2-27e7d4ad5e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:33.359475 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 133/272] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:26:33.365708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:33.380510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f84250-4386-4b12-beef-9475fc088229 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:33.380757 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f84250-4386-4b12-beef-9475fc088229 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:33.381199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f84250-4386-4b12-beef-9475fc088229 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:33.383132 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 134/273] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:26:33.389176 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320da27d-f842-488a-987e-fe472867c6fe None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:33.389658 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320da27d-f842-488a-987e-fe472867c6fe None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:33.390099 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320da27d-f842-488a-987e-fe472867c6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:33.421117 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-320da27d-f842-488a-987e-fe472867c6fe None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:33.424011 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-320da27d-f842-488a-987e-fe472867c6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:33.485246 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 140/274] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:33.487787 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe26d6-9a98-4e15-87d2-27e7d4ad5e30 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:33.568851 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 135/275] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:26:33.570759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320da27d-f842-488a-987e-fe472867c6fe None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:33.687823 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe26d6-9a98-4e15-87d2-27e7d4ad5e30 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:33.742604 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea88e6-0c0c-4f29-9427-7c78b79ce338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:33.743643 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea88e6-0c0c-4f29-9427-7c78b79ce338 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:33.743926 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea88e6-0c0c-4f29-9427-7c78b79ce338 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:33.744205 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea88e6-0c0c-4f29-9427-7c78b79ce338 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:33.746541 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320da27d-f842-488a-987e-fe472867c6fe None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:33.755007 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea88e6-0c0c-4f29-9427-7c78b79ce338 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:33.759919 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:33.762774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-67ea88e6-0c0c-4f29-9427-7c78b79ce338 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:33.762774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:33.762774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:33.762774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:33.762774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:33.762774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:33.762774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:33.762774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:33.762774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:33.762774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:33.762774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:33.762774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:33.762774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:33.762774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:33.762774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:33.762774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:33.763930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:33.763930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:33.763930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:33.763930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:33.763930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:33.763930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:33.763930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:33.763930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:33.763930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:33.763930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:33.763930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:33.763930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:33.763930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:33.763930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:33.763930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:33.763930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:33.764901 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:33.764901 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:33.764901 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:33.764901 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:33.764901 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:33.764901 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:33.764901 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:33.764901 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:33.764901 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:33.764901 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:33.764901 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:33.764901 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:33.766917 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 141/276] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:33.767855 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea88e6-0c0c-4f29-9427-7c78b79ce338 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:33.778542 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:33.790764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7a337-cb55-42a2-bc85-e8aaeffd44d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:33.791549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7a337-cb55-42a2-bc85-e8aaeffd44d6 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:33.791829 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7a337-cb55-42a2-bc85-e8aaeffd44d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:33.792103 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7a337-cb55-42a2-bc85-e8aaeffd44d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:33.793254 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7a337-cb55-42a2-bc85-e8aaeffd44d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:33.795729 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7a337-cb55-42a2-bc85-e8aaeffd44d6 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:33.798275 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 136/277] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:26:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:33.799556 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7a337-cb55-42a2-bc85-e8aaeffd44d6 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:33.880852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea88e6-0c0c-4f29-9427-7c78b79ce338 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:33.902239 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7a337-cb55-42a2-bc85-e8aaeffd44d6 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:33.918078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7863c3-aa99-4d48-b5ab-439e8f343a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:33.919033 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7863c3-aa99-4d48-b5ab-439e8f343a9c None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:33.919473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7863c3-aa99-4d48-b5ab-439e8f343a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:33.919898 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7863c3-aa99-4d48-b5ab-439e8f343a9c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:33.930795 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7863c3-aa99-4d48-b5ab-439e8f343a9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:33.938506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52eedbb-1c4c-4151-aba2-1169cb85e1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:33.939112 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52eedbb-1c4c-4151-aba2-1169cb85e1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:33.939392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52eedbb-1c4c-4151-aba2-1169cb85e1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:33.939667 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52eedbb-1c4c-4151-aba2-1169cb85e1f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:33.940034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-ef7863c3-aa99-4d48-b5ab-439e8f343a9c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:33.940034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:33.940034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:33.940034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:33.940034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:33.940034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:33.940034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:33.940034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:33.940034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:33.940034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:33.940034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:33.940034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:33.940034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:33.940034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:33.940034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:33.940034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:33.940909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:33.940909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:33.940909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:33.940909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:33.940909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:33.940909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:33.940909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:33.940909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:33.940909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:33.940909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:33.940909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:33.940909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:33.940909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:33.940909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:33.940909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:33.940909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:33.941694 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:33.941694 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:33.941694 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:33.941694 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:33.941694 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:33.941694 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:33.941694 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:33.941694 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:33.941694 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:33.941694 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:33.941694 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:33.941694 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:33.942286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52eedbb-1c4c-4151-aba2-1169cb85e1f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:33.942286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52eedbb-1c4c-4151-aba2-1169cb85e1f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:33.942286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52eedbb-1c4c-4151-aba2-1169cb85e1f6 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:33.946362 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 142/278] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:33.947577 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7863c3-aa99-4d48-b5ab-439e8f343a9c None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:33.953838 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 137/279] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:33.955378 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52eedbb-1c4c-4151-aba2-1169cb85e1f6 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:33.998361 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:34.017105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:34.051525 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7863c3-aa99-4d48-b5ab-439e8f343a9c None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:34.057612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52eedbb-1c4c-4151-aba2-1169cb85e1f6 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:34.085701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577a3df-14eb-4185-80af-9bee1c685045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:34.086361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577a3df-14eb-4185-80af-9bee1c685045 None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:34.086680 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577a3df-14eb-4185-80af-9bee1c685045 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:34.086957 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577a3df-14eb-4185-80af-9bee1c685045 None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:34.087655 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b577a3df-14eb-4185-80af-9bee1c685045 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:34.089114 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b577a3df-14eb-4185-80af-9bee1c685045 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:34.090659 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33144b6-d1c0-4784-91f2-d0c754d73509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:34.091317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33144b6-d1c0-4784-91f2-d0c754d73509 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:34.091605 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33144b6-d1c0-4784-91f2-d0c754d73509 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:34.091884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33144b6-d1c0-4784-91f2-d0c754d73509 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:34.092626 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33144b6-d1c0-4784-91f2-d0c754d73509 None admin] RBAC: Adding query filter params (name=member) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:34.092932 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33144b6-d1c0-4784-91f2-d0c754d73509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:34.094286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33144b6-d1c0-4784-91f2-d0c754d73509 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:34.102731 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 138/280] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:34.103598 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33144b6-d1c0-4784-91f2-d0c754d73509 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:34.177705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 143/281] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Jun 27 18:26:33 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:26:34.196099 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e64974-16ac-4ed8-94b1-5c6dc81f93ba None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:34.196286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e64974-16ac-4ed8-94b1-5c6dc81f93ba None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:34.196566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e64974-16ac-4ed8-94b1-5c6dc81f93ba None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:34.198125 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 144/282] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:34.202744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:34.203369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e64974-16ac-4ed8-94b1-5c6dc81f93ba None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:34.206947 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33144b6-d1c0-4784-91f2-d0c754d73509 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:34.248898 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b42aa-7782-479a-b481-51a330839f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:34.250143 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b42aa-7782-479a-b481-51a330839f68 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:34.250597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b42aa-7782-479a-b481-51a330839f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:34.251024 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b42aa-7782-479a-b481-51a330839f68 None admin] PATH_INFO: `/v3/users/649e74029eb44705b54c17a9cbc17ef7` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:34.267564 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b42aa-7782-479a-b481-51a330839f68 None admin] RBAC: Authorizing `identity:get_user(user_id=649e74029eb44705b54c17a9cbc17ef7)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:34.269078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b42aa-7782-479a-b481-51a330839f68 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:34.275642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 139/283] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:34 2022] GET /identity/v3/users/649e74029eb44705b54c17a9cbc17ef7 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:34.275642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e4f08-8d15-4237-b144-247cdca960e6 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:34.275642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e4f08-8d15-4237-b144-247cdca960e6 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:34.275642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e4f08-8d15-4237-b144-247cdca960e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:34.294239 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-ed9e4f08-8d15-4237-b144-247cdca960e6 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:34.296141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-ed9e4f08-8d15-4237-b144-247cdca960e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:34.326278 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e64974-16ac-4ed8-94b1-5c6dc81f93ba None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:34.353257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:34.368312 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ac9d5-b0ab-413c-a25c-a430fdb5c328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:34.369242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ac9d5-b0ab-413c-a25c-a430fdb5c328 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:34.369745 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ac9d5-b0ab-413c-a25c-a430fdb5c328 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:34.369873 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ac9d5-b0ab-413c-a25c-a430fdb5c328 None admin] PATH_INFO: `/v3/users/fc8e5aab0e4f496ba69a5b9461948d79` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:34.371068 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ac9d5-b0ab-413c-a25c-a430fdb5c328 None admin] RBAC: Authorizing `identity:get_user(user_id=fc8e5aab0e4f496ba69a5b9461948d79)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:34.373082 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ac9d5-b0ab-413c-a25c-a430fdb5c328 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:34.374893 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 145/284] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:34 2022] GET /identity/v3/users/fc8e5aab0e4f496ba69a5b9461948d79 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:34.375658 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ac9d5-b0ab-413c-a25c-a430fdb5c328 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:34.388170 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 140/285] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:34.388170 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e4f08-8d15-4237-b144-247cdca960e6 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:34.499458 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:34.512373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ac9d5-b0ab-413c-a25c-a430fdb5c328 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:34.513722 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e4f08-8d15-4237-b144-247cdca960e6 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:34.556603 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8098017-76d0-439e-adeb-3158448c9c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:34.557469 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8098017-76d0-439e-adeb-3158448c9c42 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:34.557849 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8098017-76d0-439e-adeb-3158448c9c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:34.558132 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8098017-76d0-439e-adeb-3158448c9c42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:34.563306 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec0f29-8964-45d3-bee8-36a8ef2de793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:34.564253 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec0f29-8964-45d3-bee8-36a8ef2de793 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:34.564784 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec0f29-8964-45d3-bee8-36a8ef2de793 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:34.565262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec0f29-8964-45d3-bee8-36a8ef2de793 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:34.566321 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec0f29-8964-45d3-bee8-36a8ef2de793 None admin] RBAC: Adding query filter params (role.id=b45aa1c36d974fe79c809afbfcd01945, user.id=649e74029eb44705b54c17a9cbc17ef7, scope.system=all) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:34.566868 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec0f29-8964-45d3-bee8-36a8ef2de793 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:34.569081 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec0f29-8964-45d3-bee8-36a8ef2de793 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:34.577432 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 146/286] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Jun 27 18:26:34 2022] GET /identity/v3/role_assignments?user.id=649e74029eb44705b54c17a9cbc17ef7&scope.system=all&role.id=b45aa1c36d974fe79c809afbfcd01945 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:34.578727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec0f29-8964-45d3-bee8-36a8ef2de793 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:34.643685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8098017-76d0-439e-adeb-3158448c9c42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:34.645488 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8098017-76d0-439e-adeb-3158448c9c42 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:34.649871 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:34.699532 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec0f29-8964-45d3-bee8-36a8ef2de793 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:34.742409 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851be212-b200-430c-bb79-489b61e6721a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:34.742409 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851be212-b200-430c-bb79-489b61e6721a None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:34.742409 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851be212-b200-430c-bb79-489b61e6721a None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:34.742409 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851be212-b200-430c-bb79-489b61e6721a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:34.750650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851be212-b200-430c-bb79-489b61e6721a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:34.758667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-851be212-b200-430c-bb79-489b61e6721a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 18:26:34.758667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:34.758667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:34.758667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:34.758667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:34.758667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:34.758667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:34.758667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:34.758667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:34.758667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:34.758667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:34.758667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:34.758667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:34.758667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:34.758667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:34.758667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:34.759562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:34.759562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:34.759562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:34.759562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:34.759562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:34.759562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:34.759562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:34.759562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:34.759562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:34.759562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:34.759562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:34.759562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:34.759562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:34.759562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:34.759562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:34.759562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:34.760672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:34.760672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:34.760672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:34.760672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:34.760672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:34.760672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:34.760672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:34.760672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:34.760672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:34.760672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:34.760672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 18:26:34.760672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:34.760672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 147/287] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:34.763305 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:34.766587 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851be212-b200-430c-bb79-489b61e6721a None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:34.787229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 141/288] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:26:34.795607 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8098017-76d0-439e-adeb-3158448c9c42 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:34.891471 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851be212-b200-430c-bb79-489b61e6721a None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:34.913861 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:34.949551 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a50617-aaad-409f-897b-3735ab369b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:34.950370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a50617-aaad-409f-897b-3735ab369b11 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:34.950766 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a50617-aaad-409f-897b-3735ab369b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:34.951114 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a50617-aaad-409f-897b-3735ab369b11 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:34.954432 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a50617-aaad-409f-897b-3735ab369b11 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:34.954432 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a50617-aaad-409f-897b-3735ab369b11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:34.954432 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a50617-aaad-409f-897b-3735ab369b11 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:34.963911 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 148/289] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:34.970232 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a50617-aaad-409f-897b-3735ab369b11 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:34.998309 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8098017-76d0-439e-adeb-3158448c9c42 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:35.063018 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86a340-7d1e-4edc-87ce-0712f139c078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:35.063974 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86a340-7d1e-4edc-87ce-0712f139c078 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:35.064258 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86a340-7d1e-4edc-87ce-0712f139c078 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:35.064532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86a340-7d1e-4edc-87ce-0712f139c078 None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:35.067460 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86a340-7d1e-4edc-87ce-0712f139c078 None admin] RBAC: Authorizing `identity:get_project(project_id=4d7f983bf3f84850a4a59357572371d9)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:35.068903 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86a340-7d1e-4edc-87ce-0712f139c078 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:35.070879 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 142/290] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:34 2022] GET /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9 => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:35.077164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86a340-7d1e-4edc-87ce-0712f139c078 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:35.156078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:35.180325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a50617-aaad-409f-897b-3735ab369b11 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:35.202985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86a340-7d1e-4edc-87ce-0712f139c078 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:35.244624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317621b8-6ffb-4c75-8d50-b47742864544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:35.245280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317621b8-6ffb-4c75-8d50-b47742864544 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:35.245564 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317621b8-6ffb-4c75-8d50-b47742864544 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:35.245840 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317621b8-6ffb-4c75-8d50-b47742864544 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:35.247210 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317621b8-6ffb-4c75-8d50-b47742864544 None admin] RBAC: Adding query filter params (role.id=b45aa1c36d974fe79c809afbfcd01945, scope.project.id=4d7f983bf3f84850a4a59357572371d9, user.id=fc8e5aab0e4f496ba69a5b9461948d79) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:35.247525 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317621b8-6ffb-4c75-8d50-b47742864544 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:35.248958 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317621b8-6ffb-4c75-8d50-b47742864544 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:35.252534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2cd8e-4c60-45d0-9331-55ca4eb9aae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:35.253421 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2cd8e-4c60-45d0-9331-55ca4eb9aae1 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:35.253705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2cd8e-4c60-45d0-9331-55ca4eb9aae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:35.253976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2cd8e-4c60-45d0-9331-55ca4eb9aae1 None admin] PATH_INFO: `/v3/users/1dc3d318f83a41a0aa4b852a15721b78` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:35.255674 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 143/291] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:26:35 2022] GET /identity/v3/role_assignments?user.id=fc8e5aab0e4f496ba69a5b9461948d79&scope.project.id=4d7f983bf3f84850a4a59357572371d9&role.id=b45aa1c36d974fe79c809afbfcd01945 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:35.256404 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2cd8e-4c60-45d0-9331-55ca4eb9aae1 None admin] RBAC: Authorizing `identity:get_user(user_id=1dc3d318f83a41a0aa4b852a15721b78)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:35.257878 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2cd8e-4c60-45d0-9331-55ca4eb9aae1 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:35.259742 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 149/292] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:34 2022] GET /identity/v3/users/1dc3d318f83a41a0aa4b852a15721b78 => generated 297 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:35.265357 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317621b8-6ffb-4c75-8d50-b47742864544 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:35.386597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317621b8-6ffb-4c75-8d50-b47742864544 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:35.421164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:35.444571 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae193f-59f6-4a8a-a1a3-8759bc6cdb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:35.445464 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae193f-59f6-4a8a-a1a3-8759bc6cdb75 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:35.445744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae193f-59f6-4a8a-a1a3-8759bc6cdb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:35.446022 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae193f-59f6-4a8a-a1a3-8759bc6cdb75 None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:35.447295 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae193f-59f6-4a8a-a1a3-8759bc6cdb75 None admin] RBAC: Authorizing `identity:get_project(project_id=4d7f983bf3f84850a4a59357572371d9)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:35.448771 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae193f-59f6-4a8a-a1a3-8759bc6cdb75 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:35.451609 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 144/293] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:35 2022] GET /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:35.457212 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2cd8e-4c60-45d0-9331-55ca4eb9aae1 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:35.518983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:35.580850 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2cd8e-4c60-45d0-9331-55ca4eb9aae1 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:35.616133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fbd9f3-6ca5-46e9-86ef-18524c52e1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:35.617022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbd9f3-6ca5-46e9-86ef-18524c52e1b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:35.617305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbd9f3-6ca5-46e9-86ef-18524c52e1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:35.617581 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbd9f3-6ca5-46e9-86ef-18524c52e1b3 None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9/users/1dc3d318f83a41a0aa4b852a15721b78/roles/b042cc08697840f5a9ae4066a6f0205c` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:35.619593 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fbd9f3-6ca5-46e9-86ef-18524c52e1b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d7f983bf3f84850a4a59357572371d9, user_id=1dc3d318f83a41a0aa4b852a15721b78, role_id=b042cc08697840f5a9ae4066a6f0205c)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:35.621045 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fbd9f3-6ca5-46e9-86ef-18524c52e1b3 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:35.693087 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 150/294] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:26:35 2022] PUT /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9/users/1dc3d318f83a41a0aa4b852a15721b78/roles/b042cc08697840f5a9ae4066a6f0205c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:26:36.499011 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2323b20-cdd2-4623-a783-41828942d453 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:36.499526 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2323b20-cdd2-4623-a783-41828942d453 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:36.499601 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2323b20-cdd2-4623-a783-41828942d453 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:36.501168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 145/295] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:36.506559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f89cf6-3a04-49fc-8abc-a2b849232ccc None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:36.507379 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f89cf6-3a04-49fc-8abc-a2b849232ccc None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:36.507859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f89cf6-3a04-49fc-8abc-a2b849232ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:36.524821 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-10f89cf6-3a04-49fc-8abc-a2b849232ccc None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:36.526327 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-10f89cf6-3a04-49fc-8abc-a2b849232ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:36.601209 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 151/296] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:36.624877 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2323b20-cdd2-4623-a783-41828942d453 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:36.720164 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297c23c-e00c-4c61-b425-e767e053a96e None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:36.720500 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297c23c-e00c-4c61-b425-e767e053a96e None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:36.720748 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297c23c-e00c-4c61-b425-e767e053a96e None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:36.723263 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 152/297] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:36.729489 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed813c9-308e-4b06-be5a-1947df5a0bc6 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:36.730021 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed813c9-308e-4b06-be5a-1947df5a0bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:36.730540 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed813c9-308e-4b06-be5a-1947df5a0bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:36.739071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2323b20-cdd2-4623-a783-41828942d453 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:36.750091 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-9ed813c9-308e-4b06-be5a-1947df5a0bc6 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:36.751251 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-9ed813c9-308e-4b06-be5a-1947df5a0bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:36.780221 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:36.785564 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb3110-a894-46d9-be23-c39b59bdad21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:36.786355 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb3110-a894-46d9-be23-c39b59bdad21 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:36.786660 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb3110-a894-46d9-be23-c39b59bdad21 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:36.786931 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb3110-a894-46d9-be23-c39b59bdad21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:36.788194 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb3110-a894-46d9-be23-c39b59bdad21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:36.789645 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb3110-a894-46d9-be23-c39b59bdad21 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:36.791563 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 146/298] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:26:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:36.798863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb3110-a894-46d9-be23-c39b59bdad21 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:36.834585 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 153/299] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:26:36.859867 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed813c9-308e-4b06-be5a-1947df5a0bc6 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:36.873093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:36.924082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb3110-a894-46d9-be23-c39b59bdad21 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:36.960690 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8e73d-9934-40bb-9fc0-7ad6c6a4fa97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:36.961668 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8e73d-9934-40bb-9fc0-7ad6c6a4fa97 None admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:36.962219 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8e73d-9934-40bb-9fc0-7ad6c6a4fa97 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:36.962532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8e73d-9934-40bb-9fc0-7ad6c6a4fa97 None admin] PATH_INFO: `/v3/groups` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:36.963224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8e73d-9934-40bb-9fc0-7ad6c6a4fa97 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:36.964860 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc8e73d-9934-40bb-9fc0-7ad6c6a4fa97 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:36.968067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed813c9-308e-4b06-be5a-1947df5a0bc6 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:37.003300 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018b112-0eb7-4080-af64-2f2fd447f651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:37.004023 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018b112-0eb7-4080-af64-2f2fd447f651 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:37.004332 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018b112-0eb7-4080-af64-2f2fd447f651 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:37.004614 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018b112-0eb7-4080-af64-2f2fd447f651 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:37.014570 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018b112-0eb7-4080-af64-2f2fd447f651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:37.022280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-8018b112-0eb7-4080-af64-2f2fd447f651 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 18:26:37.022280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:37.022280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:37.022280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:37.022280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:37.022280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:37.022280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:37.022280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:37.022280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:37.022280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:37.022280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:37.022280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:37.022280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:37.022280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:37.022280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:37.022280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:37.023369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:37.023369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:37.023369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:37.023369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:37.023369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:37.023369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:37.023369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:37.023369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:37.023369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:37.023369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:37.023369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:37.023369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:37.023369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:37.023369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:37.023369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:37.023369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:37.029108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:37.029108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:37.029108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:37.029108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:37.029108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:37.029108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:37.029108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:37.029108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:37.029108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:37.029108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:37.029108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 18:26:37.029108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:37.029108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 154/300] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:37.029108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf56b4a-933d-463e-be89-d658fcff8cb2 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:37.029845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf56b4a-933d-463e-be89-d658fcff8cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:37.029845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf56b4a-933d-463e-be89-d658fcff8cb2 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:37.029845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 155/301] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:37.031217 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf56b4a-933d-463e-be89-d658fcff8cb2 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:37.056206 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:37.065677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 147/302] 10.222.0.15 () {60 vars in 1134 bytes} [Mon Jun 27 18:26:36 2022] POST /identity/v3/groups => generated 229 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:26:37.067025 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8c78d-4f87-4928-9389-f9d4a8bdb263 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:37.067317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8c78d-4f87-4928-9389-f9d4a8bdb263 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:37.067606 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8c78d-4f87-4928-9389-f9d4a8bdb263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:37.092898 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-1fe8c78d-4f87-4928-9389-f9d4a8bdb263 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:37.094155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-1fe8c78d-4f87-4928-9389-f9d4a8bdb263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:37.157225 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf56b4a-933d-463e-be89-d658fcff8cb2 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:37.187375 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 148/303] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:37.200248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d497b965-f730-4ca2-867e-4fd6b0556673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:37.201579 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497b965-f730-4ca2-867e-4fd6b0556673 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:37.202049 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497b965-f730-4ca2-867e-4fd6b0556673 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:37.202533 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497b965-f730-4ca2-867e-4fd6b0556673 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:37.203519 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d497b965-f730-4ca2-867e-4fd6b0556673 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:37.204067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d497b965-f730-4ca2-867e-4fd6b0556673 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:37.205742 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d497b965-f730-4ca2-867e-4fd6b0556673 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:37.210637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8c78d-4f87-4928-9389-f9d4a8bdb263 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:37.217601 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 156/304] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:37.227695 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d497b965-f730-4ca2-867e-4fd6b0556673 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:37.277667 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:37.415228 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:37.428185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8c78d-4f87-4928-9389-f9d4a8bdb263 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:37.435649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d497b965-f730-4ca2-867e-4fd6b0556673 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:37.468023 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8efb38-7e98-4600-b44f-534537fb4a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:37.469287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8efb38-7e98-4600-b44f-534537fb4a49 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:37.469767 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8efb38-7e98-4600-b44f-534537fb4a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:37.470306 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8efb38-7e98-4600-b44f-534537fb4a49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:37.477602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86eb19-d0c6-4418-be24-3ce0a30496b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:37.478494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86eb19-d0c6-4418-be24-3ce0a30496b5 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:37.478830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86eb19-d0c6-4418-be24-3ce0a30496b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:37.479033 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86eb19-d0c6-4418-be24-3ce0a30496b5 None admin] PATH_INFO: `/v3/users/e192acd9c5434e639508e7ff666143a1` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:37.484358 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8efb38-7e98-4600-b44f-534537fb4a49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:37.492895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-ae8efb38-7e98-4600-b44f-534537fb4a49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:37.492895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:37.492895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:37.492895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:37.492895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:37.492895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:37.492895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:37.492895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:37.492895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:37.492895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:37.492895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:37.492895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:37.492895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:37.492895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:37.492895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:37.492895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:37.493774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:37.493774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:37.493774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:37.493774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:37.493774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:37.493774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:37.493774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:37.493774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:37.493774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:37.493774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:37.493774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:37.493774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:37.493774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:37.493774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:37.493774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:37.493774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:37.494617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:37.494617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:37.494617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:37.494617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:37.494617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:37.494617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:37.494617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:37.494617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:37.494617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:37.494617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:37.494617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:37.494617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:26:37.496017 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 149/305] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:26:37.496543 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86eb19-d0c6-4418-be24-3ce0a30496b5 None admin] RBAC: Authorizing `identity:get_user(user_id=e192acd9c5434e639508e7ff666143a1)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:37.502482 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86eb19-d0c6-4418-be24-3ce0a30496b5 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:37.502482 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 157/306] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:37 2022] GET /identity/v3/users/e192acd9c5434e639508e7ff666143a1 => generated 293 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:37.505986 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86eb19-d0c6-4418-be24-3ce0a30496b5 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:37.506110 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8efb38-7e98-4600-b44f-534537fb4a49 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:37.625271 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86eb19-d0c6-4418-be24-3ce0a30496b5 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:37.641971 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8efb38-7e98-4600-b44f-534537fb4a49 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:37.651996 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:37.665441 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75720998-c7d5-4d31-8529-51b364b505e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:37.666159 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75720998-c7d5-4d31-8529-51b364b505e2 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:37.666566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75720998-c7d5-4d31-8529-51b364b505e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:37.667102 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75720998-c7d5-4d31-8529-51b364b505e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:37.667879 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75720998-c7d5-4d31-8529-51b364b505e2 None admin] RBAC: Adding query filter params (role.id=b042cc08697840f5a9ae4066a6f0205c, user.id=e192acd9c5434e639508e7ff666143a1, scope.system=all) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:37.668326 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75720998-c7d5-4d31-8529-51b364b505e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:37.671316 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75720998-c7d5-4d31-8529-51b364b505e2 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:37.682827 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 158/307] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Jun 27 18:26:37 2022] GET /identity/v3/role_assignments?user.id=e192acd9c5434e639508e7ff666143a1&scope.system=all&role.id=b042cc08697840f5a9ae4066a6f0205c => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:37.687547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6336119-064a-4a9b-85d1-c111590f9bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:37.688452 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6336119-064a-4a9b-85d1-c111590f9bcd None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:37.688881 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6336119-064a-4a9b-85d1-c111590f9bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:37.689293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6336119-064a-4a9b-85d1-c111590f9bcd None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:37.690252 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6336119-064a-4a9b-85d1-c111590f9bcd None admin] RBAC: Adding query filter params (name=member) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:37.690763 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6336119-064a-4a9b-85d1-c111590f9bcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:37.692326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6336119-064a-4a9b-85d1-c111590f9bcd None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:37.707829 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 150/308] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:37.709507 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75720998-c7d5-4d31-8529-51b364b505e2 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:37.722213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:37.783014 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfed36-7778-4691-9081-14f0ba53aac3 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:37.783294 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfed36-7778-4691-9081-14f0ba53aac3 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:37.783571 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfed36-7778-4691-9081-14f0ba53aac3 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:37.785214 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 151/309] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:37.790570 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d184c2c-0f3d-4af7-8ee6-b7869ce9301e None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:37.790852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d184c2c-0f3d-4af7-8ee6-b7869ce9301e None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:37.791128 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d184c2c-0f3d-4af7-8ee6-b7869ce9301e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:37.809125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-0d184c2c-0f3d-4af7-8ee6-b7869ce9301e None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:37.810897 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-0d184c2c-0f3d-4af7-8ee6-b7869ce9301e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:37.848480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75720998-c7d5-4d31-8529-51b364b505e2 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:37.885868 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 152/310] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:26:37.895046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359983d5-6f81-47a3-bf95-b0e325eef456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:37.896114 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359983d5-6f81-47a3-bf95-b0e325eef456 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:37.896543 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359983d5-6f81-47a3-bf95-b0e325eef456 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:37.896990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359983d5-6f81-47a3-bf95-b0e325eef456 None admin] PATH_INFO: `/v3/users/649e74029eb44705b54c17a9cbc17ef7` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:37.898341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359983d5-6f81-47a3-bf95-b0e325eef456 None admin] RBAC: Authorizing `identity:get_user(user_id=649e74029eb44705b54c17a9cbc17ef7)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:37.899947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359983d5-6f81-47a3-bf95-b0e325eef456 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:37.906415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 159/311] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:37 2022] GET /identity/v3/users/649e74029eb44705b54c17a9cbc17ef7 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:37.909320 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359983d5-6f81-47a3-bf95-b0e325eef456 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:37.909460 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d184c2c-0f3d-4af7-8ee6-b7869ce9301e None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:38.019358 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:38.021581 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:38.026759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d184c2c-0f3d-4af7-8ee6-b7869ce9301e None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:38.043730 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359983d5-6f81-47a3-bf95-b0e325eef456 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:38.077451 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679e438-b812-4474-bc9d-0a25081248b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:38.078779 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679e438-b812-4474-bc9d-0a25081248b8 None admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:38.079209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679e438-b812-4474-bc9d-0a25081248b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:38.079321 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefd480-3171-4c96-a4e2-463cedde0210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:38.079733 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679e438-b812-4474-bc9d-0a25081248b8 None admin] PATH_INFO: `/v3/system/users/649e74029eb44705b54c17a9cbc17ef7/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:38.080053 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefd480-3171-4c96-a4e2-463cedde0210 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:38.080336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefd480-3171-4c96-a4e2-463cedde0210 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:38.080608 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefd480-3171-4c96-a4e2-463cedde0210 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:38.081858 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a679e438-b812-4474-bc9d-0a25081248b8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=649e74029eb44705b54c17a9cbc17ef7, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:38.083918 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a679e438-b812-4474-bc9d-0a25081248b8 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:38.091772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aefd480-3171-4c96-a4e2-463cedde0210 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:38.099924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-4aefd480-3171-4c96-a4e2-463cedde0210 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 18:26:38.099924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:38.099924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:38.099924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:38.099924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:38.099924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:38.099924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:38.099924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:38.099924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:38.099924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:38.099924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:38.099924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:38.099924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:38.099924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:38.099924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:38.099924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:38.100859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:38.100859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:38.100859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:38.100859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:38.100859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:38.100859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:38.100859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:38.100859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:38.100859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:38.100859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:38.100859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:38.100859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:38.100859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:38.100859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:38.100859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:38.100859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:38.101710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:38.101710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:38.101710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:38.101710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:38.101710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:38.101710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:38.101710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:38.101710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:38.101710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:38.101710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:38.101710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 18:26:38.101710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:38.101710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 160/312] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:26:38.108361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefd480-3171-4c96-a4e2-463cedde0210 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:38.185415 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 153/313] 10.222.0.15 () {58 vars in 1334 bytes} [Mon Jun 27 18:26:37 2022] PUT /identity/v3/system/users/649e74029eb44705b54c17a9cbc17ef7/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:26:38.221312 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefd480-3171-4c96-a4e2-463cedde0210 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:38.263455 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28459985-b688-4576-8914-7e6c64bf7cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:38.264475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28459985-b688-4576-8914-7e6c64bf7cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:38.264919 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28459985-b688-4576-8914-7e6c64bf7cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:38.265385 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28459985-b688-4576-8914-7e6c64bf7cf3 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:38.266486 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28459985-b688-4576-8914-7e6c64bf7cf3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:38.267037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28459985-b688-4576-8914-7e6c64bf7cf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:38.268719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28459985-b688-4576-8914-7e6c64bf7cf3 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:38.281158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 161/314] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:38.281515 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:38.286566 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679e438-b812-4474-bc9d-0a25081248b8 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:38.399500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679e438-b812-4474-bc9d-0a25081248b8 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:38.435747 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a079c-d4d4-4477-91e0-328f6faa94cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:38.436536 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a079c-d4d4-4477-91e0-328f6faa94cf None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:38.436815 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a079c-d4d4-4477-91e0-328f6faa94cf None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:38.437082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a079c-d4d4-4477-91e0-328f6faa94cf None admin] PATH_INFO: `/v3/users/1dc3d318f83a41a0aa4b852a15721b78` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:38.438285 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a079c-d4d4-4477-91e0-328f6faa94cf None admin] RBAC: Authorizing `identity:get_user(user_id=1dc3d318f83a41a0aa4b852a15721b78)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:38.439786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a079c-d4d4-4477-91e0-328f6faa94cf None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:38.441715 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 154/315] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:38 2022] GET /identity/v3/users/1dc3d318f83a41a0aa4b852a15721b78 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:38.446937 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28459985-b688-4576-8914-7e6c64bf7cf3 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:38.556212 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28459985-b688-4576-8914-7e6c64bf7cf3 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:38.591651 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5edf40-6457-4539-b779-ff16452a9175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:38.592299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5edf40-6457-4539-b779-ff16452a9175 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:38.592583 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5edf40-6457-4539-b779-ff16452a9175 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:38.592859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5edf40-6457-4539-b779-ff16452a9175 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:38.662814 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:38.664619 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5edf40-6457-4539-b779-ff16452a9175 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:38.666320 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5edf40-6457-4539-b779-ff16452a9175 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:38.780113 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 162/316] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:26:38.785189 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a079c-d4d4-4477-91e0-328f6faa94cf None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:38.817720 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:38.895899 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a079c-d4d4-4477-91e0-328f6faa94cf None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:38.932270 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69252a-b0f4-45e8-b58b-bf54955c13b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:38.933492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69252a-b0f4-45e8-b58b-bf54955c13b1 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:38.933968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69252a-b0f4-45e8-b58b-bf54955c13b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:38.935003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69252a-b0f4-45e8-b58b-bf54955c13b1 None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:38.936418 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69252a-b0f4-45e8-b58b-bf54955c13b1 None admin] RBAC: Authorizing `identity:get_project(project_id=4d7f983bf3f84850a4a59357572371d9)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:38.938036 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69252a-b0f4-45e8-b58b-bf54955c13b1 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:38.940258 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 155/317] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:38 2022] GET /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:38.946259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5edf40-6457-4539-b779-ff16452a9175 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:39.050119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:39.055624 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5edf40-6457-4539-b779-ff16452a9175 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:39.093168 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c7b25-d865-4ecd-b230-ca5cc2e71eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:39.093821 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c7b25-d865-4ecd-b230-ca5cc2e71eda None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:39.094616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c7b25-d865-4ecd-b230-ca5cc2e71eda None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:39.094892 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c7b25-d865-4ecd-b230-ca5cc2e71eda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:39.095712 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c7b25-d865-4ecd-b230-ca5cc2e71eda None admin] RBAC: Adding query filter params (role.id=b042cc08697840f5a9ae4066a6f0205c, scope.project.id=4d7f983bf3f84850a4a59357572371d9, user.id=1dc3d318f83a41a0aa4b852a15721b78) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:39.096018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c7b25-d865-4ecd-b230-ca5cc2e71eda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:39.097441 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91c7b25-d865-4ecd-b230-ca5cc2e71eda None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:39.103391 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 163/318] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:26:38 2022] GET /identity/v3/role_assignments?user.id=1dc3d318f83a41a0aa4b852a15721b78&scope.project.id=4d7f983bf3f84850a4a59357572371d9&role.id=b042cc08697840f5a9ae4066a6f0205c => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:39.224193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d42060-5981-448f-92a9-111bd86c6b35 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:39.224529 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d42060-5981-448f-92a9-111bd86c6b35 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:39.224740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d42060-5981-448f-92a9-111bd86c6b35 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:39.226302 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 156/319] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:39.233270 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a53a6-cdf2-4474-8fb8-dbd34d7df4f2 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:39.233909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a53a6-cdf2-4474-8fb8-dbd34d7df4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:39.234413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a53a6-cdf2-4474-8fb8-dbd34d7df4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:39.251921 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-d47a53a6-cdf2-4474-8fb8-dbd34d7df4f2 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:39.253656 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-d47a53a6-cdf2-4474-8fb8-dbd34d7df4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:39.328261 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 164/320] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:39.352227 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d42060-5981-448f-92a9-111bd86c6b35 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:39.455186 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:39.460280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d42060-5981-448f-92a9-111bd86c6b35 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:39.495235 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd9d867-3881-4d64-839e-9302350da8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:39.496019 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9d867-3881-4d64-839e-9302350da8d2 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:39.496297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9d867-3881-4d64-839e-9302350da8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:39.496663 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9d867-3881-4d64-839e-9302350da8d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:39.498008 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd9d867-3881-4d64-839e-9302350da8d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:39.499478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd9d867-3881-4d64-839e-9302350da8d2 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:39.501306 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 157/321] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:26:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:39.506715 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a53a6-cdf2-4474-8fb8-dbd34d7df4f2 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:39.596407 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:39.605438 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a53a6-cdf2-4474-8fb8-dbd34d7df4f2 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:39.639798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64f48d-cb43-4f1d-8ee9-2cf83c1b2025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:39.640458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64f48d-cb43-4f1d-8ee9-2cf83c1b2025 None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:39.640741 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64f48d-cb43-4f1d-8ee9-2cf83c1b2025 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:39.641018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64f48d-cb43-4f1d-8ee9-2cf83c1b2025 None admin] PATH_INFO: `/v3/groups` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:39.641701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64f48d-cb43-4f1d-8ee9-2cf83c1b2025 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:39.643209 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64f48d-cb43-4f1d-8ee9-2cf83c1b2025 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:39.716645 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 165/322] 10.222.0.15 () {60 vars in 1134 bytes} [Mon Jun 27 18:26:39 2022] POST /identity/v3/groups => generated 226 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:26:39.837957 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a245f0b-ae69-4b0d-81c3-fffd08ac9754 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:39.838658 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a245f0b-ae69-4b0d-81c3-fffd08ac9754 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:39.838658 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a245f0b-ae69-4b0d-81c3-fffd08ac9754 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:39.840139 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 158/323] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:39.846115 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a2365-5597-4453-b8a5-ec4f95900dfb None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:39.846497 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a2365-5597-4453-b8a5-ec4f95900dfb None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:39.846725 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a2365-5597-4453-b8a5-ec4f95900dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:39.864552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-c48a2365-5597-4453-b8a5-ec4f95900dfb None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:39.865932 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c48a2365-5597-4453-b8a5-ec4f95900dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:39.938285 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 166/324] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:39.961443 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a245f0b-ae69-4b0d-81c3-fffd08ac9754 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:40.074949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a245f0b-ae69-4b0d-81c3-fffd08ac9754 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:40.112827 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4cd0d-0f08-434e-973d-57932a98b777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:40.113681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4cd0d-0f08-434e-973d-57932a98b777 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:40.113935 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4cd0d-0f08-434e-973d-57932a98b777 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:40.114265 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4cd0d-0f08-434e-973d-57932a98b777 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:40.126289 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4cd0d-0f08-434e-973d-57932a98b777 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:40.134303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-ecf4cd0d-0f08-434e-973d-57932a98b777 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 18:26:40.134303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:40.134303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:40.134303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:40.134303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:40.134303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:40.134303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:40.134303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:40.134303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:40.134303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:40.134303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:40.134303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:40.134303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:40.134303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:40.134303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:40.134303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:40.135321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:40.135321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:40.135321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:40.135321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:40.135321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:40.135321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:40.135321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:40.135321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:40.135321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:40.135321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:40.135321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:40.135321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:40.135321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:40.135321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:40.135321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:40.135321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:40.136371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:40.136371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:40.136371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:40.136371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:40.136371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:40.136371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:40.136371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:40.136371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:40.136371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:40.136371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:40.136371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 18:26:40.136371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:26:40.136371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 159/325] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:40.143437 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a2365-5597-4453-b8a5-ec4f95900dfb None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:40.250751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:40.271600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a2365-5597-4453-b8a5-ec4f95900dfb None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:40.310313 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b59e24-b1e2-4152-addd-d74dc75097c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:40.311045 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b59e24-b1e2-4152-addd-d74dc75097c2 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:40.311331 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b59e24-b1e2-4152-addd-d74dc75097c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:40.311608 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b59e24-b1e2-4152-addd-d74dc75097c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:40.312366 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b59e24-b1e2-4152-addd-d74dc75097c2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:40.312668 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b59e24-b1e2-4152-addd-d74dc75097c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:40.314090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b59e24-b1e2-4152-addd-d74dc75097c2 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:40.325542 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 167/326] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:40.330134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4cd0d-0f08-434e-973d-57932a98b777 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:40.363483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:40.401552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5391c9-8ae9-4a18-a073-110206b467f6 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:40.401861 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5391c9-8ae9-4a18-a073-110206b467f6 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:40.402122 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5391c9-8ae9-4a18-a073-110206b467f6 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:40.405433 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 168/327] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:26:40.411067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4701247-0c48-44af-ba8e-5b0823c3bd6e None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:40.411495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4701247-0c48-44af-ba8e-5b0823c3bd6e None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:40.411872 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4701247-0c48-44af-ba8e-5b0823c3bd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:40.438226 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-f4701247-0c48-44af-ba8e-5b0823c3bd6e None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:40.439750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-f4701247-0c48-44af-ba8e-5b0823c3bd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:40.470168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4cd0d-0f08-434e-973d-57932a98b777 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:40.515802 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd5aa2-2e36-448e-b48a-ff1f1743f93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:40.516765 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd5aa2-2e36-448e-b48a-ff1f1743f93f None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:40.517048 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd5aa2-2e36-448e-b48a-ff1f1743f93f None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:40.517318 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd5aa2-2e36-448e-b48a-ff1f1743f93f None admin] PATH_INFO: `/v3/users/e192acd9c5434e639508e7ff666143a1` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:40.520492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd5aa2-2e36-448e-b48a-ff1f1743f93f None admin] RBAC: Authorizing `identity:get_user(user_id=e192acd9c5434e639508e7ff666143a1)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:40.520492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd5aa2-2e36-448e-b48a-ff1f1743f93f None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:40.526832 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 160/328] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:40 2022] GET /identity/v3/users/e192acd9c5434e639508e7ff666143a1 => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:40.529463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd5aa2-2e36-448e-b48a-ff1f1743f93f None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:40.532403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 169/329] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:26:40.557288 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4701247-0c48-44af-ba8e-5b0823c3bd6e None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:40.655137 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd5aa2-2e36-448e-b48a-ff1f1743f93f None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:40.656702 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:40.674753 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:40.686339 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4701247-0c48-44af-ba8e-5b0823c3bd6e None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:40.715122 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025088f-1341-44ac-8350-e30de11456a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:40.716323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025088f-1341-44ac-8350-e30de11456a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:40.716605 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025088f-1341-44ac-8350-e30de11456a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:40.716881 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025088f-1341-44ac-8350-e30de11456a9 None admin] PATH_INFO: `/v3/system/users/e192acd9c5434e639508e7ff666143a1/roles/b042cc08697840f5a9ae4066a6f0205c` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:40.718481 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025088f-1341-44ac-8350-e30de11456a9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e192acd9c5434e639508e7ff666143a1, role_id=b042cc08697840f5a9ae4066a6f0205c)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:40.720042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025088f-1341-44ac-8350-e30de11456a9 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:40.722509 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b8320-e1db-44ad-b6e9-ce00d41fa552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:40.723314 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b8320-e1db-44ad-b6e9-ce00d41fa552 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:40.723605 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b8320-e1db-44ad-b6e9-ce00d41fa552 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:40.723888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b8320-e1db-44ad-b6e9-ce00d41fa552 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:40.738430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b8320-e1db-44ad-b6e9-ce00d41fa552 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:40.747086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-810b8320-e1db-44ad-b6e9-ce00d41fa552 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:40.747086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:40.747086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:40.747086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:40.747086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:40.747086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:40.747086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:40.747086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:40.747086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:40.747086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:40.747086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:40.747086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:40.747086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:40.747086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:40.747086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:40.747086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:40.748208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:40.748208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:40.748208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:40.748208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:40.748208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:40.748208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:40.748208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:40.748208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:40.748208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:40.748208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:40.748208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:40.748208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:40.748208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:40.748208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:40.748208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:40.748208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:40.749602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:40.749602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:40.749602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:40.749602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:40.749602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:40.749602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:40.749602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:40.749602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:40.749602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:40.749602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:40.749602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:40.749602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:40.749602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 170/330] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:26:40.754516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b8320-e1db-44ad-b6e9-ce00d41fa552 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:40.788991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 161/331] 10.222.0.15 () {58 vars in 1334 bytes} [Mon Jun 27 18:26:40 2022] PUT /identity/v3/system/users/e192acd9c5434e639508e7ff666143a1/roles/b042cc08697840f5a9ae4066a6f0205c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:26:40.869268 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b8320-e1db-44ad-b6e9-ce00d41fa552 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:40.909075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9c589-40c7-434b-9b7e-5eb805030eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:40.909680 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9c589-40c7-434b-9b7e-5eb805030eee None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:40.909967 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9c589-40c7-434b-9b7e-5eb805030eee None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:40.910244 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9c589-40c7-434b-9b7e-5eb805030eee None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:40.911065 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc9c589-40c7-434b-9b7e-5eb805030eee None admin] RBAC: Adding query filter params (name=member) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:40.911377 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc9c589-40c7-434b-9b7e-5eb805030eee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:40.912830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc9c589-40c7-434b-9b7e-5eb805030eee None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:40.914489 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:40.923593 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 171/332] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:40.929357 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025088f-1341-44ac-8350-e30de11456a9 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:41.054480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025088f-1341-44ac-8350-e30de11456a9 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:41.093431 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b54337-0890-46c8-b10c-1eb191f5827d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:41.094319 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b54337-0890-46c8-b10c-1eb191f5827d None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:41.094635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b54337-0890-46c8-b10c-1eb191f5827d None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:41.094909 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b54337-0890-46c8-b10c-1eb191f5827d None admin] PATH_INFO: `/v3/users/649e74029eb44705b54c17a9cbc17ef7` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:41.096146 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b54337-0890-46c8-b10c-1eb191f5827d None admin] RBAC: Authorizing `identity:get_user(user_id=649e74029eb44705b54c17a9cbc17ef7)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:41.097725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b54337-0890-46c8-b10c-1eb191f5827d None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:41.099707 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 162/333] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:40 2022] GET /identity/v3/users/649e74029eb44705b54c17a9cbc17ef7 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:41.105430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9c589-40c7-434b-9b7e-5eb805030eee None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:41.222550 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9c589-40c7-434b-9b7e-5eb805030eee None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:41.259215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf25d7-76db-46f2-9c35-c1c4ed969c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:41.259824 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf25d7-76db-46f2-9c35-c1c4ed969c61 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:41.260107 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf25d7-76db-46f2-9c35-c1c4ed969c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:41.264089 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf25d7-76db-46f2-9c35-c1c4ed969c61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:41.264089 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf25d7-76db-46f2-9c35-c1c4ed969c61 None admin] RBAC: Adding query filter params (role.id=b45aa1c36d974fe79c809afbfcd01945, user.id=649e74029eb44705b54c17a9cbc17ef7, scope.system=all) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:41.264089 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf25d7-76db-46f2-9c35-c1c4ed969c61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:41.264089 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf25d7-76db-46f2-9c35-c1c4ed969c61 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:41.275866 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 172/334] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Jun 27 18:26:41 2022] GET /identity/v3/role_assignments?user.id=649e74029eb44705b54c17a9cbc17ef7&scope.system=all&role.id=b45aa1c36d974fe79c809afbfcd01945 => generated 510 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:42.398369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05506b43-11c5-48d4-b6e8-7ce1f9144ae0 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:42.398904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05506b43-11c5-48d4-b6e8-7ce1f9144ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:42.398979 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05506b43-11c5-48d4-b6e8-7ce1f9144ae0 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:42.400472 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 163/335] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:42.406610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc4155-44f2-4205-9a06-88d21332653b None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:42.407446 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc4155-44f2-4205-9a06-88d21332653b None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:42.407931 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc4155-44f2-4205-9a06-88d21332653b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:42.426523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-d5dc4155-44f2-4205-9a06-88d21332653b None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:42.428054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-d5dc4155-44f2-4205-9a06-88d21332653b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:42.485607 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:42.508269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 173/336] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:42.546122 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05506b43-11c5-48d4-b6e8-7ce1f9144ae0 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:42.581658 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:42.674875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05506b43-11c5-48d4-b6e8-7ce1f9144ae0 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:42.721527 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c9c53-30e6-4f82-9499-036a7aec232b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:42.722321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c9c53-30e6-4f82-9499-036a7aec232b None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:42.722628 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c9c53-30e6-4f82-9499-036a7aec232b None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:42.722900 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c9c53-30e6-4f82-9499-036a7aec232b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:42.732212 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aaca26-5cca-430c-aa24-06b29acc0617 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:42.732510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aaca26-5cca-430c-aa24-06b29acc0617 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:42.732837 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aaca26-5cca-430c-aa24-06b29acc0617 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:42.743371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c9c53-30e6-4f82-9499-036a7aec232b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:42.751745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-2e2c9c53-30e6-4f82-9499-036a7aec232b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:42.751745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:42.751745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:42.751745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:42.751745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:42.751745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:42.751745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:42.751745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:42.751745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:42.751745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:42.751745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:42.751745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:42.751745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:42.751745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:42.751745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:42.751745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:42.752636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:42.752636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:42.752636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:42.752636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:42.752636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:42.752636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:42.752636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:42.752636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:42.752636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:42.752636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:42.752636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:42.752636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:42.752636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:42.752636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:42.752636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:42.752636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:42.753440 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:42.753440 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:42.753440 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:42.753440 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:42.753440 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:42.753440 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:42.753440 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:42.753440 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:42.753440 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:42.753440 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:42.753440 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:42.753440 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:26:42.753440 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 164/338] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:42.754098 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 174/338] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:26:42.759187 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c9c53-30e6-4f82-9499-036a7aec232b None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:42.760251 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e26a7-07d6-4132-96fe-486ef1d9280c None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:42.760544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e26a7-07d6-4132-96fe-486ef1d9280c None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:42.760821 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e26a7-07d6-4132-96fe-486ef1d9280c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:42.790078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-853e26a7-07d6-4132-96fe-486ef1d9280c None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:42.791363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-853e26a7-07d6-4132-96fe-486ef1d9280c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:42.942292 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 175/339] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:26:42.952412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:42.963912 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c9c53-30e6-4f82-9499-036a7aec232b None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:42.992137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e26a7-07d6-4132-96fe-486ef1d9280c None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:43.029253 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d6d0a-9620-4357-b00c-c10d8839f914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:43.029925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d6d0a-9620-4357-b00c-c10d8839f914 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:43.030209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d6d0a-9620-4357-b00c-c10d8839f914 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:43.030513 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d6d0a-9620-4357-b00c-c10d8839f914 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:43.031271 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7d6d0a-9620-4357-b00c-c10d8839f914 None admin] RBAC: Adding query filter params (name=member) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:43.031578 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7d6d0a-9620-4357-b00c-c10d8839f914 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:43.034887 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7d6d0a-9620-4357-b00c-c10d8839f914 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:43.058409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 165/340] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:43.061700 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d6d0a-9620-4357-b00c-c10d8839f914 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:43.162494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e26a7-07d6-4132-96fe-486ef1d9280c None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:43.190771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:43.212420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5a421-1d50-482b-8068-a7aa56b62344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:43.212420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5a421-1d50-482b-8068-a7aa56b62344 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:43.212420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5a421-1d50-482b-8068-a7aa56b62344 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:43.212420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c5a421-1d50-482b-8068-a7aa56b62344 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:43.228971 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c5a421-1d50-482b-8068-a7aa56b62344 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:43.238939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-72c5a421-1d50-482b-8068-a7aa56b62344 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:43.238939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:43.238939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:43.238939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:43.238939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:43.238939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:43.238939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:43.238939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:43.238939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:43.238939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:43.238939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:43.238939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:43.238939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:43.238939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:43.238939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:43.238939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:43.239854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:43.239854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:43.239854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:43.239854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:43.239854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:43.239854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:43.239854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:43.239854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:43.239854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:43.239854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:43.239854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:43.239854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:43.239854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:43.239854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:43.239854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:43.239854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:43.240631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:43.240631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:43.240631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:43.240631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:43.240631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:43.240631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:43.240631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:43.240631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:43.240631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:43.240631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:43.240631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:43.240631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:43.244574 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 176/341] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Jun 27 18:26:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:43.248047 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5a421-1d50-482b-8068-a7aa56b62344 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:43.256967 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d6d0a-9620-4357-b00c-c10d8839f914 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:43.321957 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bda413-1e86-446b-81fa-eecc35e23a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:43.322628 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bda413-1e86-446b-81fa-eecc35e23a02 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:43.322910 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bda413-1e86-446b-81fa-eecc35e23a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:43.323401 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bda413-1e86-446b-81fa-eecc35e23a02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:43.370750 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:43.443301 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bda413-1e86-446b-81fa-eecc35e23a02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:43.444756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bda413-1e86-446b-81fa-eecc35e23a02 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:43.483650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c5a421-1d50-482b-8068-a7aa56b62344 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:43.555077 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 166/342] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 18:26:43.561900 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bda413-1e86-446b-81fa-eecc35e23a02 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:43.571104 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde13a3-ca21-4251-b93c-a6e677a97256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:43.571770 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde13a3-ca21-4251-b93c-a6e677a97256 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:43.572050 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde13a3-ca21-4251-b93c-a6e677a97256 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:43.572323 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde13a3-ca21-4251-b93c-a6e677a97256 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:43.573077 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde13a3-ca21-4251-b93c-a6e677a97256 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:43.573380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde13a3-ca21-4251-b93c-a6e677a97256 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:43.577931 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde13a3-ca21-4251-b93c-a6e677a97256 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:43.594161 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 177/343] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:26:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:43.600509 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde13a3-ca21-4251-b93c-a6e677a97256 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:43.656436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:43.671090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:43.744319 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bda413-1e86-446b-81fa-eecc35e23a02 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:43.765344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde13a3-ca21-4251-b93c-a6e677a97256 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:43.781557 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea8d54-57f6-47f0-a48b-5919ab3e03d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:43.782715 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea8d54-57f6-47f0-a48b-5919ab3e03d7 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:43.783134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea8d54-57f6-47f0-a48b-5919ab3e03d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:43.783547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea8d54-57f6-47f0-a48b-5919ab3e03d7 None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:43.784871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea8d54-57f6-47f0-a48b-5919ab3e03d7 None admin] RBAC: Authorizing `identity:get_project(project_id=d1f168a334d74b6bb18fa28c3f9774d1)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:43.786540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea8d54-57f6-47f0-a48b-5919ab3e03d7 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:43.788697 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 167/344] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:43 2022] GET /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:43.790068 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b16f4-b59a-4334-a852-fb967ec428ab None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:43.790518 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b16f4-b59a-4334-a852-fb967ec428ab None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:43.790932 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b16f4-b59a-4334-a852-fb967ec428ab None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:43.792508 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 168/345] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:43.794217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b16f4-b59a-4334-a852-fb967ec428ab None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:43.807164 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb56583-83e8-41f1-97b3-f467b8d77ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:43.808053 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb56583-83e8-41f1-97b3-f467b8d77ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:43.808487 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb56583-83e8-41f1-97b3-f467b8d77ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:43.808969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb56583-83e8-41f1-97b3-f467b8d77ba1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:43.886207 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb56583-83e8-41f1-97b3-f467b8d77ba1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:43.888585 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb56583-83e8-41f1-97b3-f467b8d77ba1 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:43.909303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b16f4-b59a-4334-a852-fb967ec428ab None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:43.927753 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:43.944158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:43.948868 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f673200-dfad-404d-bd45-361e617d4a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:43.950075 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f673200-dfad-404d-bd45-361e617d4a5c None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:43.950550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f673200-dfad-404d-bd45-361e617d4a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:43.950970 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f673200-dfad-404d-bd45-361e617d4a5c None admin] PATH_INFO: `/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:43.957402 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f673200-dfad-404d-bd45-361e617d4a5c None admin] RBAC: Authorizing `identity:get_group(group_id=9f7e220e2ba14938ae0cac70c096cb8b)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:43.959061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f673200-dfad-404d-bd45-361e617d4a5c None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:43.961364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 169/346] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Jun 27 18:26:43 2022] GET /identity/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:43.962867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdb2cd-91ec-4460-99db-99e851c93307 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:43.963293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdb2cd-91ec-4460-99db-99e851c93307 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:43.963705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdb2cd-91ec-4460-99db-99e851c93307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:43.983930 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-a5bdb2cd-91ec-4460-99db-99e851c93307 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:43.985685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-a5bdb2cd-91ec-4460-99db-99e851c93307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:44.000012 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 178/347] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:26:44.000857 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb56583-83e8-41f1-97b3-f467b8d77ba1 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:44.062096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 170/348] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:44.067156 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdb2cd-91ec-4460-99db-99e851c93307 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:44.114225 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb56583-83e8-41f1-97b3-f467b8d77ba1 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:44.163041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50be573-c430-40a6-86df-22cbb9001cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:44.163940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50be573-c430-40a6-86df-22cbb9001cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:44.164363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50be573-c430-40a6-86df-22cbb9001cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:44.164777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50be573-c430-40a6-86df-22cbb9001cc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:44.165850 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50be573-c430-40a6-86df-22cbb9001cc8 None admin] RBAC: Adding query filter params (group.id=9f7e220e2ba14938ae0cac70c096cb8b, role.id=b45aa1c36d974fe79c809afbfcd01945, scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:44.166303 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50be573-c430-40a6-86df-22cbb9001cc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:44.168022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50be573-c430-40a6-86df-22cbb9001cc8 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:44.174831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:44.180285 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 179/349] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Jun 27 18:26:43 2022] GET /identity/v3/role_assignments?group.id=9f7e220e2ba14938ae0cac70c096cb8b&scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1&role.id=b45aa1c36d974fe79c809afbfcd01945 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:44.181273 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50be573-c430-40a6-86df-22cbb9001cc8 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:44.195394 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdb2cd-91ec-4460-99db-99e851c93307 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:44.210573 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:44.235278 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad63895-e291-4e30-9f01-a2602c12b40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:44.236277 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad63895-e291-4e30-9f01-a2602c12b40d None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:44.236633 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad63895-e291-4e30-9f01-a2602c12b40d None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:44.237048 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad63895-e291-4e30-9f01-a2602c12b40d None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:44.238472 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad63895-e291-4e30-9f01-a2602c12b40d None admin] RBAC: Authorizing `identity:get_project(project_id=d1f168a334d74b6bb18fa28c3f9774d1)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:44.240015 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad63895-e291-4e30-9f01-a2602c12b40d None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:44.242475 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 171/350] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:44 2022] GET /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:44.243819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d211aa-6e8a-4636-a1c7-a8f6cf67e1a8 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:44.244105 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d211aa-6e8a-4636-a1c7-a8f6cf67e1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:44.244396 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d211aa-6e8a-4636-a1c7-a8f6cf67e1a8 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:44.245633 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 172/351] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:44.248320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d211aa-6e8a-4636-a1c7-a8f6cf67e1a8 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:44.365546 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d211aa-6e8a-4636-a1c7-a8f6cf67e1a8 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:44.380262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50be573-c430-40a6-86df-22cbb9001cc8 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:44.403904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a64934-1d93-4cf8-9314-92a25747f692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:44.404777 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a64934-1d93-4cf8-9314-92a25747f692 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:44.405055 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a64934-1d93-4cf8-9314-92a25747f692 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:44.405337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a64934-1d93-4cf8-9314-92a25747f692 None admin] PATH_INFO: `/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:44.406585 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a64934-1d93-4cf8-9314-92a25747f692 None admin] RBAC: Authorizing `identity:get_group(group_id=9f7e220e2ba14938ae0cac70c096cb8b)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:44.408108 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a64934-1d93-4cf8-9314-92a25747f692 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:44.410009 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 173/352] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Jun 27 18:26:44 2022] GET /identity/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:44.411185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7840e1-5bb1-4933-9278-50fe784c5f82 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:44.411468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7840e1-5bb1-4933-9278-50fe784c5f82 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:44.411746 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7840e1-5bb1-4933-9278-50fe784c5f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:44.429196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-6f7840e1-5bb1-4933-9278-50fe784c5f82 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:44.430621 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-6f7840e1-5bb1-4933-9278-50fe784c5f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:44.434245 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28fed4-210e-4d96-95b2-c742ab5cdd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:44.435339 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28fed4-210e-4d96-95b2-c742ab5cdd94 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:44.435774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28fed4-210e-4d96-95b2-c742ab5cdd94 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:44.436194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28fed4-210e-4d96-95b2-c742ab5cdd94 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:44.456786 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d28fed4-210e-4d96-95b2-c742ab5cdd94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:44.473665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-7d28fed4-210e-4d96-95b2-c742ab5cdd94 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:44.473665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:44.473665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:44.473665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:44.473665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:44.473665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:44.473665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:44.473665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:44.473665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:44.473665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:44.473665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:44.473665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:44.473665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:44.473665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:44.473665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:44.473665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:44.474853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:44.474853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:44.474853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:44.474853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:44.474853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:44.474853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:44.474853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:44.474853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:44.474853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:44.474853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:44.474853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:44.474853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:44.474853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:44.474853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:44.474853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:44.474853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:44.475848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:44.475848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:44.475848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:44.475848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:44.475848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:44.475848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:44.475848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:44.475848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:44.475848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:44.475848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:44.475848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:44.475848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:44.477538 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:44.478272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 180/353] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:44.479262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28fed4-210e-4d96-95b2-c742ab5cdd94 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:44.508269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 174/354] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:44.508911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7840e1-5bb1-4933-9278-50fe784c5f82 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:44.573765 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:44.624803 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28fed4-210e-4d96-95b2-c742ab5cdd94 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:44.649572 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7840e1-5bb1-4933-9278-50fe784c5f82 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:44.661169 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3bd58-04d5-4488-b18f-89e03738bf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:44.662114 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3bd58-04d5-4488-b18f-89e03738bf81 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:44.662638 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3bd58-04d5-4488-b18f-89e03738bf81 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:44.663114 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3bd58-04d5-4488-b18f-89e03738bf81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:44.664147 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3bd58-04d5-4488-b18f-89e03738bf81 None admin] RBAC: Adding query filter params (group.id=9f7e220e2ba14938ae0cac70c096cb8b, role.id=44d7d6bfe7eb441bbc6a301a5b6784ad, scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:44.664697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3bd58-04d5-4488-b18f-89e03738bf81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:44.666347 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3bd58-04d5-4488-b18f-89e03738bf81 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:44.673993 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 181/355] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Jun 27 18:26:44 2022] GET /identity/v3/role_assignments?group.id=9f7e220e2ba14938ae0cac70c096cb8b&scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1&role.id=44d7d6bfe7eb441bbc6a301a5b6784ad => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:44.675617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9abee2-5d40-42b3-9ecf-c4127f663ff0 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:44.676107 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9abee2-5d40-42b3-9ecf-c4127f663ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:44.676602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9abee2-5d40-42b3-9ecf-c4127f663ff0 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:44.678249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 182/356] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:44.679248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9abee2-5d40-42b3-9ecf-c4127f663ff0 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:44.703567 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bed98-c124-4662-92c9-8b2f5bcd5d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:44.704262 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bed98-c124-4662-92c9-8b2f5bcd5d66 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:44.704510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bed98-c124-4662-92c9-8b2f5bcd5d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:44.704790 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bed98-c124-4662-92c9-8b2f5bcd5d66 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:44.705554 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bed98-c124-4662-92c9-8b2f5bcd5d66 None admin] RBAC: Adding query filter params (name=member) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:44.705862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bed98-c124-4662-92c9-8b2f5bcd5d66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:44.707266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bed98-c124-4662-92c9-8b2f5bcd5d66 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:44.725232 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 175/357] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:44.725625 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08d1d7-b13c-4268-acbc-0688697795ed None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:44.725917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08d1d7-b13c-4268-acbc-0688697795ed None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:44.726260 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08d1d7-b13c-4268-acbc-0688697795ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:44.775532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-1d08d1d7-b13c-4268-acbc-0688697795ed None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:44.776746 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-1d08d1d7-b13c-4268-acbc-0688697795ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:44.805389 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9abee2-5d40-42b3-9ecf-c4127f663ff0 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:44.822453 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:44.850414 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:44.850414 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff7ebc-9009-46d3-bcd3-afadcaa5d124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:44.850414 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff7ebc-9009-46d3-bcd3-afadcaa5d124 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:44.850414 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff7ebc-9009-46d3-bcd3-afadcaa5d124 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:44.850939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff7ebc-9009-46d3-bcd3-afadcaa5d124 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:44.866413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff7ebc-9009-46d3-bcd3-afadcaa5d124 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:44.877317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-50ff7ebc-9009-46d3-bcd3-afadcaa5d124 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:44.877317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:44.877317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:44.877317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:44.877317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:44.877317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:44.877317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:44.877317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:44.877317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:44.877317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:44.877317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:44.877317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:44.877317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:44.877317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:44.877317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:44.877317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:44.878277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:44.878277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:44.878277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:44.878277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:44.878277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:44.878277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:44.878277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:44.878277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:44.878277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:44.878277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:44.878277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:44.878277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:44.878277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:44.878277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:44.878277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:44.878277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:44.879105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:44.879105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:44.879105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:44.879105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:44.879105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:44.879105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:44.879105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:44.879105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:44.879105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:44.879105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:44.879105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:44.879105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:44.879105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 183/358] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Jun 27 18:26:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:44.879105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff7ebc-9009-46d3-bcd3-afadcaa5d124 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:44.894130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 176/359] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:44.895320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08d1d7-b13c-4268-acbc-0688697795ed None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:45.028898 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff7ebc-9009-46d3-bcd3-afadcaa5d124 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:45.052067 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08d1d7-b13c-4268-acbc-0688697795ed None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:45.082679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d78a7-fec6-4dae-a751-24986f6d0022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:45.083329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d78a7-fec6-4dae-a751-24986f6d0022 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:45.083609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d78a7-fec6-4dae-a751-24986f6d0022 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:45.083881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d78a7-fec6-4dae-a751-24986f6d0022 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:45.099013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7258c61-1df9-4f1f-a932-d377397fb386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:45.099013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7258c61-1df9-4f1f-a932-d377397fb386 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:45.099270 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7258c61-1df9-4f1f-a932-d377397fb386 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:45.099528 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7258c61-1df9-4f1f-a932-d377397fb386 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:45.100280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7258c61-1df9-4f1f-a932-d377397fb386 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:45.100581 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7258c61-1df9-4f1f-a932-d377397fb386 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:45.102125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7258c61-1df9-4f1f-a932-d377397fb386 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:45.112964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 177/360] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:26:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:45.113627 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7258c61-1df9-4f1f-a932-d377397fb386 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:45.153171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:45.156990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:45.174352 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d78a7-fec6-4dae-a751-24986f6d0022 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:45.175998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d78a7-fec6-4dae-a751-24986f6d0022 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:45.249570 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7258c61-1df9-4f1f-a932-d377397fb386 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:45.289921 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 184/361] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:26:45.291527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35c70f-ced4-4a09-8533-3bbf1fb02700 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:45.291816 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35c70f-ced4-4a09-8533-3bbf1fb02700 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:45.292152 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35c70f-ced4-4a09-8533-3bbf1fb02700 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:45.294355 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 185/362] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:45.296654 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35c70f-ced4-4a09-8533-3bbf1fb02700 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:45.311587 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f386ba2-4890-44ff-8ca7-1c52f3e053e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:45.312388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f386ba2-4890-44ff-8ca7-1c52f3e053e0 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:45.312668 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f386ba2-4890-44ff-8ca7-1c52f3e053e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:45.312942 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f386ba2-4890-44ff-8ca7-1c52f3e053e0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:45.331534 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f386ba2-4890-44ff-8ca7-1c52f3e053e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:45.338478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-7f386ba2-4890-44ff-8ca7-1c52f3e053e0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 18:26:45.338478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:45.338478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:45.338478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:45.338478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:45.338478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:45.338478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:45.338478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:45.338478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:45.338478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:45.338478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:45.338478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:45.338478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:45.338478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:45.338478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:45.338478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:45.339374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:45.339374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:45.339374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:45.339374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:45.339374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:45.339374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:45.339374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:45.339374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:45.339374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:45.339374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:45.339374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:45.339374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:45.339374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:45.339374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:45.339374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:45.339374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:45.340229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:45.340229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:45.340229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:45.340229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:45.340229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:45.340229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:45.340229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:45.340229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:45.340229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:45.340229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:45.340229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 18:26:45.340229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:26:45.340229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 178/363] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:45.340883 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f386ba2-4890-44ff-8ca7-1c52f3e053e0 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:45.439030 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:45.452111 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:45.460907 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35c70f-ced4-4a09-8533-3bbf1fb02700 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:45.462515 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f386ba2-4890-44ff-8ca7-1c52f3e053e0 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:45.513485 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b36b34-583b-433f-8974-51cc431ebcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:45.514638 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b36b34-583b-433f-8974-51cc431ebcec None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:45.515005 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b36b34-583b-433f-8974-51cc431ebcec None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:45.515400 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b36b34-583b-433f-8974-51cc431ebcec None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:45.515901 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794aca16-b23c-4fb6-858e-8fbb172845e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:45.516775 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794aca16-b23c-4fb6-858e-8fbb172845e5 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:45.516886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b36b34-583b-433f-8974-51cc431ebcec None admin] RBAC: Authorizing `identity:get_project(project_id=4d7f983bf3f84850a4a59357572371d9)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:45.517362 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794aca16-b23c-4fb6-858e-8fbb172845e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:45.517835 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794aca16-b23c-4fb6-858e-8fbb172845e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:45.518453 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b36b34-583b-433f-8974-51cc431ebcec None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:45.520758 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 179/364] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:45 2022] GET /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:45.522037 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7d0d5-3cdc-4c0a-898c-b45af1cc4cda None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:45.522321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7d0d5-3cdc-4c0a-898c-b45af1cc4cda None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:45.522620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7d0d5-3cdc-4c0a-898c-b45af1cc4cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:45.543391 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-f6e7d0d5-3cdc-4c0a-898c-b45af1cc4cda None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:45.544890 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f6e7d0d5-3cdc-4c0a-898c-b45af1cc4cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:45.613195 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794aca16-b23c-4fb6-858e-8fbb172845e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:45.615441 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794aca16-b23c-4fb6-858e-8fbb172845e5 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:45.635355 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 180/365] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:45.636269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7d0d5-3cdc-4c0a-898c-b45af1cc4cda None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:45.742461 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:45.769296 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:45.772536 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 186/366] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:26:45.773265 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794aca16-b23c-4fb6-858e-8fbb172845e5 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:45.790762 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7d0d5-3cdc-4c0a-898c-b45af1cc4cda None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:45.843682 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4a9b7-3d44-415c-8a09-410a4c065270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:45.844864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4a9b7-3d44-415c-8a09-410a4c065270 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:45.845318 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4a9b7-3d44-415c-8a09-410a4c065270 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:45.846161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4a9b7-3d44-415c-8a09-410a4c065270 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:45.847410 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a4a9b7-3d44-415c-8a09-410a4c065270 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:45.848127 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a4a9b7-3d44-415c-8a09-410a4c065270 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:45.850397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a4a9b7-3d44-415c-8a09-410a4c065270 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:45.865706 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 181/367] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:45.866376 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4a9b7-3d44-415c-8a09-410a4c065270 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:45.905743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794aca16-b23c-4fb6-858e-8fbb172845e5 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:45.956130 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469f690-7e5f-48b0-82c5-8fb4f80503be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:45.957084 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469f690-7e5f-48b0-82c5-8fb4f80503be None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:45.957370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469f690-7e5f-48b0-82c5-8fb4f80503be None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:45.957646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469f690-7e5f-48b0-82c5-8fb4f80503be None admin] PATH_INFO: `/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:45.958920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7469f690-7e5f-48b0-82c5-8fb4f80503be None admin] RBAC: Authorizing `identity:get_group(group_id=9f7e220e2ba14938ae0cac70c096cb8b)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:45.960651 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7469f690-7e5f-48b0-82c5-8fb4f80503be None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:45.965777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 187/368] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Jun 27 18:26:45 2022] GET /identity/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:45.966360 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469f690-7e5f-48b0-82c5-8fb4f80503be None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:46.010795 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4a9b7-3d44-415c-8a09-410a4c065270 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:46.034443 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:46.052710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ee701-d5a8-4ba3-a731-ac27e63c89b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:46.053594 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ee701-d5a8-4ba3-a731-ac27e63c89b5 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:46.053868 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ee701-d5a8-4ba3-a731-ac27e63c89b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:46.054140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ee701-d5a8-4ba3-a731-ac27e63c89b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:46.067576 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ee701-d5a8-4ba3-a731-ac27e63c89b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:46.075654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-9a9ee701-d5a8-4ba3-a731-ac27e63c89b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:26:46.075654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:46.075654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:46.075654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:46.075654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:46.075654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:46.075654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:46.075654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:46.075654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:46.075654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:46.075654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:46.075654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:46.075654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:46.075654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:46.075654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:46.075654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:46.076602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:46.076602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:46.076602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:46.076602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:46.076602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:46.076602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:46.076602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:46.076602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:46.076602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:46.076602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:46.076602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:46.076602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:46.076602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:46.076602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:46.076602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:46.076602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:46.077450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:46.077450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:46.077450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:46.077450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:46.077450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:46.077450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:46.077450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:46.077450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:46.077450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:46.077450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:46.077450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:26:46.077450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:26:46.077450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 182/369] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:26:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:46.078165 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ee701-d5a8-4ba3-a731-ac27e63c89b5 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:46.090418 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469f690-7e5f-48b0-82c5-8fb4f80503be None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:46.103405 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:46.129686 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db041be0-6fac-4166-b693-6ec52c71906f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:46.130647 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db041be0-6fac-4166-b693-6ec52c71906f None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:46.130930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db041be0-6fac-4166-b693-6ec52c71906f None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:46.131202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db041be0-6fac-4166-b693-6ec52c71906f None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:46.132464 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db041be0-6fac-4166-b693-6ec52c71906f None admin] RBAC: Authorizing `identity:get_project(project_id=4d7f983bf3f84850a4a59357572371d9)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:46.133975 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db041be0-6fac-4166-b693-6ec52c71906f None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:46.136006 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 188/370] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:45 2022] GET /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:46.136851 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db041be0-6fac-4166-b693-6ec52c71906f None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:46.231524 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ee701-d5a8-4ba3-a731-ac27e63c89b5 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:46.254916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db041be0-6fac-4166-b693-6ec52c71906f None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:46.283228 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:46.285365 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082db9d-99cd-4590-ac0e-dc8bbe56ad45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:46.286739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082db9d-99cd-4590-ac0e-dc8bbe56ad45 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:46.287558 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082db9d-99cd-4590-ac0e-dc8bbe56ad45 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:46.288321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082db9d-99cd-4590-ac0e-dc8bbe56ad45 None admin] PATH_INFO: `/v3/users/e192acd9c5434e639508e7ff666143a1` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:46.290506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f082db9d-99cd-4590-ac0e-dc8bbe56ad45 None admin] RBAC: Authorizing `identity:get_user(user_id=e192acd9c5434e639508e7ff666143a1)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:46.292889 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f082db9d-99cd-4590-ac0e-dc8bbe56ad45 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:46.296750 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 183/371] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:26:46 2022] GET /identity/v3/users/e192acd9c5434e639508e7ff666143a1 => generated 293 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:46.304317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082db9d-99cd-4590-ac0e-dc8bbe56ad45 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:46.305302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5dc5fb-4dd3-4c79-835e-b057d6eb5b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:46.306064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5dc5fb-4dd3-4c79-835e-b057d6eb5b4e None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:46.306455 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5dc5fb-4dd3-4c79-835e-b057d6eb5b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:46.306806 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5dc5fb-4dd3-4c79-835e-b057d6eb5b4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:46.307714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5dc5fb-4dd3-4c79-835e-b057d6eb5b4e None admin] RBAC: Adding query filter params (group.id=9f7e220e2ba14938ae0cac70c096cb8b, role.id=b45aa1c36d974fe79c809afbfcd01945, scope.project.id=4d7f983bf3f84850a4a59357572371d9) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:46.308097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5dc5fb-4dd3-4c79-835e-b057d6eb5b4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:46.310168 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5dc5fb-4dd3-4c79-835e-b057d6eb5b4e None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:46.317144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 189/372] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Jun 27 18:26:46 2022] GET /identity/v3/role_assignments?group.id=9f7e220e2ba14938ae0cac70c096cb8b&scope.project.id=4d7f983bf3f84850a4a59357572371d9&role.id=b45aa1c36d974fe79c809afbfcd01945 => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:46.318033 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5dc5fb-4dd3-4c79-835e-b057d6eb5b4e None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:46.454240 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5dc5fb-4dd3-4c79-835e-b057d6eb5b4e None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:46.504204 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4fec2b-3fe3-4694-967b-00fd8f6d0bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:46.505086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fec2b-3fe3-4694-967b-00fd8f6d0bcd None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:46.505368 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fec2b-3fe3-4694-967b-00fd8f6d0bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:46.505649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fec2b-3fe3-4694-967b-00fd8f6d0bcd None admin] PATH_INFO: `/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:46.511223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4fec2b-3fe3-4694-967b-00fd8f6d0bcd None admin] RBAC: Authorizing `identity:get_group(group_id=9f7e220e2ba14938ae0cac70c096cb8b)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:46.513256 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4fec2b-3fe3-4694-967b-00fd8f6d0bcd None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:46.515047 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 190/373] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Jun 27 18:26:46 2022] GET /identity/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:46.515828 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4fec2b-3fe3-4694-967b-00fd8f6d0bcd None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:46.518726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:46.547601 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082db9d-99cd-4590-ac0e-dc8bbe56ad45 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:46.586543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e2be5-f1f4-49b4-b779-3ed840d68786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:46.587583 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e2be5-f1f4-49b4-b779-3ed840d68786 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:46.588066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e2be5-f1f4-49b4-b779-3ed840d68786 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:46.588558 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e2be5-f1f4-49b4-b779-3ed840d68786 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:46.589469 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e2be5-f1f4-49b4-b779-3ed840d68786 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:46.589932 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e2be5-f1f4-49b4-b779-3ed840d68786 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:46.599875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e2be5-f1f4-49b4-b779-3ed840d68786 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:46.609435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 184/374] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:46.610769 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:46.611821 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e2be5-f1f4-49b4-b779-3ed840d68786 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:46.682986 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4fec2b-3fe3-4694-967b-00fd8f6d0bcd None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:46.759643 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6a2a6-58a4-42d1-ba22-eb00f5ebf468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:46.759805 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6a2a6-58a4-42d1-ba22-eb00f5ebf468 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:46.760068 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6a2a6-58a4-42d1-ba22-eb00f5ebf468 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:46.760345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6a2a6-58a4-42d1-ba22-eb00f5ebf468 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:46.761191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6a2a6-58a4-42d1-ba22-eb00f5ebf468 None admin] RBAC: Adding query filter params (role.id=b042cc08697840f5a9ae4066a6f0205c, user.id=e192acd9c5434e639508e7ff666143a1, scope.system=all) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:46.761504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6a2a6-58a4-42d1-ba22-eb00f5ebf468 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:46.763605 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6a2a6-58a4-42d1-ba22-eb00f5ebf468 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:46.767752 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e2be5-f1f4-49b4-b779-3ed840d68786 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:46.782688 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 191/375] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Jun 27 18:26:46 2022] GET /identity/v3/role_assignments?user.id=e192acd9c5434e639508e7ff666143a1&scope.system=all&role.id=b042cc08697840f5a9ae4066a6f0205c => generated 510 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:46.783717 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6a2a6-58a4-42d1-ba22-eb00f5ebf468 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:46.811384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f6939-22b2-4f34-9318-c3caec38bba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:46.812419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f6939-22b2-4f34-9318-c3caec38bba9 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:46.812934 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f6939-22b2-4f34-9318-c3caec38bba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:46.813468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f6939-22b2-4f34-9318-c3caec38bba9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:46.814590 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f6939-22b2-4f34-9318-c3caec38bba9 None admin] RBAC: Adding query filter params (group.id=9f7e220e2ba14938ae0cac70c096cb8b, role.id=44d7d6bfe7eb441bbc6a301a5b6784ad, scope.project.id=4d7f983bf3f84850a4a59357572371d9) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:46.815129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f6939-22b2-4f34-9318-c3caec38bba9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:46.816920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f6939-22b2-4f34-9318-c3caec38bba9 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:46.827730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 185/376] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Jun 27 18:26:46 2022] GET /identity/v3/role_assignments?group.id=9f7e220e2ba14938ae0cac70c096cb8b&scope.project.id=4d7f983bf3f84850a4a59357572371d9&role.id=44d7d6bfe7eb441bbc6a301a5b6784ad => generated 260 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:46.931855 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6a2a6-58a4-42d1-ba22-eb00f5ebf468 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:46.945968 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:46.984768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b3127-4210-4142-9367-77b70679d14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:46.985445 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b3127-4210-4142-9367-77b70679d14c None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:46.985728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b3127-4210-4142-9367-77b70679d14c None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:46.986004 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b3127-4210-4142-9367-77b70679d14c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:47.041099 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d13b8-964b-446b-a4bf-02be8b11dc66 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:47.041784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d13b8-964b-446b-a4bf-02be8b11dc66 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:47.042264 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d13b8-964b-446b-a4bf-02be8b11dc66 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:47.044031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 186/377] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:26:47.049838 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d835335a-2f34-4a6b-a1b5-5b8785a473e9 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:47.050119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d835335a-2f34-4a6b-a1b5-5b8785a473e9 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:47.050430 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d835335a-2f34-4a6b-a1b5-5b8785a473e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:47.059308 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:47.070944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-d835335a-2f34-4a6b-a1b5-5b8785a473e9 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:47.072116 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-d835335a-2f34-4a6b-a1b5-5b8785a473e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:47.081668 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b3127-4210-4142-9367-77b70679d14c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:47.087428 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b3127-4210-4142-9367-77b70679d14c None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:47.189182 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 187/378] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:26:47.219326 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 192/379] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:26:47.222778 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d835335a-2f34-4a6b-a1b5-5b8785a473e9 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:47.224586 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b3127-4210-4142-9367-77b70679d14c None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:47.238779 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:47.347140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b3127-4210-4142-9367-77b70679d14c None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:47.388627 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183877a0-5331-4f02-91d7-cd62e03b74d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:47.390446 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183877a0-5331-4f02-91d7-cd62e03b74d7 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:47.390446 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183877a0-5331-4f02-91d7-cd62e03b74d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:47.390446 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183877a0-5331-4f02-91d7-cd62e03b74d7 None admin] PATH_INFO: `/v3/projects/d9ffe361a6094e199bb35c633199d02f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:47.391575 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183877a0-5331-4f02-91d7-cd62e03b74d7 None admin] RBAC: Authorizing `identity:get_project(project_id=d9ffe361a6094e199bb35c633199d02f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:47.393496 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183877a0-5331-4f02-91d7-cd62e03b74d7 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:47.395571 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 193/380] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:47 2022] GET /identity/v3/projects/d9ffe361a6094e199bb35c633199d02f => generated 344 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:47.401668 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183877a0-5331-4f02-91d7-cd62e03b74d7 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:47.433269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d835335a-2f34-4a6b-a1b5-5b8785a473e9 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:47.502336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80237d-667b-4054-b6b9-b8728f885781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:47.503530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80237d-667b-4054-b6b9-b8728f885781 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:47.504032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80237d-667b-4054-b6b9-b8728f885781 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:47.504598 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80237d-667b-4054-b6b9-b8728f885781 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:47.506767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:47.509178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:47.518270 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80237d-667b-4054-b6b9-b8728f885781 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:47.520222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183877a0-5331-4f02-91d7-cd62e03b74d7 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:47.527921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-da80237d-667b-4054-b6b9-b8728f885781 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:47.527921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:47.527921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:47.527921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:47.527921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:47.527921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:47.527921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:47.527921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:47.527921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:47.527921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:47.527921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:47.527921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:47.527921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:47.527921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:47.527921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:47.527921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:47.529168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:47.529168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:47.529168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:47.529168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:47.529168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:47.529168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:47.529168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:47.529168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:47.529168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:47.529168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:47.529168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:47.529168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:47.529168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:47.529168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:47.529168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:47.529168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:47.529950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:47.529950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:47.529950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:47.529950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:47.529950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:47.529950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:47.529950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:47.529950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:47.529950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:47.529950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:47.529950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:47.529950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:26:47.533284 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 188/381] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:47.534881 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e421f-69a1-4c64-9c9b-d82044e599f5 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:47.535380 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e421f-69a1-4c64-9c9b-d82044e599f5 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:47.535863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e421f-69a1-4c64-9c9b-d82044e599f5 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:47.537645 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 189/382] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:47.542784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e687a4b-2203-435d-94b6-914fe4d03f81 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:47.543862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e687a4b-2203-435d-94b6-914fe4d03f81 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:47.544290 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e687a4b-2203-435d-94b6-914fe4d03f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:47.561590 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf42bd0-625e-42bc-b51e-10899c00a11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:47.563222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf42bd0-625e-42bc-b51e-10899c00a11c None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:47.563222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf42bd0-625e-42bc-b51e-10899c00a11c None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:47.563222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf42bd0-625e-42bc-b51e-10899c00a11c None admin] PATH_INFO: `/v3/groups/6e27a65d60a144a88caed67204d4a9bb` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:47.568199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-3e687a4b-2203-435d-94b6-914fe4d03f81 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:47.569585 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-3e687a4b-2203-435d-94b6-914fe4d03f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:47.570130 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf42bd0-625e-42bc-b51e-10899c00a11c None admin] RBAC: Authorizing `identity:get_group(group_id=6e27a65d60a144a88caed67204d4a9bb)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:47.571822 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf42bd0-625e-42bc-b51e-10899c00a11c None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:47.574891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 194/383] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Jun 27 18:26:47 2022] GET /identity/v3/groups/6e27a65d60a144a88caed67204d4a9bb => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:47.579669 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf42bd0-625e-42bc-b51e-10899c00a11c None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:47.655995 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 190/384] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:26:47.657697 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e687a4b-2203-435d-94b6-914fe4d03f81 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:47.708687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf42bd0-625e-42bc-b51e-10899c00a11c None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:47.754897 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28aa07-6923-41e1-8b80-266cd65425cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:47.756330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28aa07-6923-41e1-8b80-266cd65425cc None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:47.756786 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28aa07-6923-41e1-8b80-266cd65425cc None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:47.757207 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28aa07-6923-41e1-8b80-266cd65425cc None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:47.758104 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28aa07-6923-41e1-8b80-266cd65425cc None admin] RBAC: Adding query filter params (name=member) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:47.759021 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28aa07-6923-41e1-8b80-266cd65425cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:47.761111 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28aa07-6923-41e1-8b80-266cd65425cc None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:47.773019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 195/385] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:47.776301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28aa07-6923-41e1-8b80-266cd65425cc None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:47.794539 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e687a4b-2203-435d-94b6-914fe4d03f81 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:47.804688 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:47.825840 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:47.840633 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d1de0-abb9-462d-9175-3c9372fc2e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:47.842022 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d1de0-abb9-462d-9175-3c9372fc2e69 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:47.842521 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d1de0-abb9-462d-9175-3c9372fc2e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:47.843288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d1de0-abb9-462d-9175-3c9372fc2e69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:47.844564 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d1de0-abb9-462d-9175-3c9372fc2e69 None admin] RBAC: Adding query filter params (group.id=6e27a65d60a144a88caed67204d4a9bb, role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f, scope.project.id=d9ffe361a6094e199bb35c633199d02f) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:47.845393 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d1de0-abb9-462d-9175-3c9372fc2e69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:47.847354 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d1de0-abb9-462d-9175-3c9372fc2e69 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:47.854625 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 191/386] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Jun 27 18:26:47 2022] GET /identity/v3/role_assignments?group.id=6e27a65d60a144a88caed67204d4a9bb&scope.project.id=d9ffe361a6094e199bb35c633199d02f&role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:47.857065 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d1de0-abb9-462d-9175-3c9372fc2e69 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:47.948560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28aa07-6923-41e1-8b80-266cd65425cc None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:47.983458 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d1de0-abb9-462d-9175-3c9372fc2e69 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:48.026070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd098662-2933-4e0d-88dc-059c18a8e61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:48.027282 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd098662-2933-4e0d-88dc-059c18a8e61c None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:48.027806 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd098662-2933-4e0d-88dc-059c18a8e61c None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:48.028476 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd098662-2933-4e0d-88dc-059c18a8e61c None admin] PATH_INFO: `/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:48.029972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd098662-2933-4e0d-88dc-059c18a8e61c None admin] RBAC: Authorizing `identity:get_group(group_id=9f7e220e2ba14938ae0cac70c096cb8b)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:48.032177 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd098662-2933-4e0d-88dc-059c18a8e61c None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:48.034398 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 192/387] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Jun 27 18:26:47 2022] GET /identity/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:48.044558 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd098662-2933-4e0d-88dc-059c18a8e61c None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:48.063114 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9808a7-6364-447c-96ce-0f4f7003fa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:48.064029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9808a7-6364-447c-96ce-0f4f7003fa55 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:48.064318 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9808a7-6364-447c-96ce-0f4f7003fa55 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:48.064596 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9808a7-6364-447c-96ce-0f4f7003fa55 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:48.091475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9808a7-6364-447c-96ce-0f4f7003fa55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:48.099845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-cb9808a7-6364-447c-96ce-0f4f7003fa55 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:48.099845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:48.099845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:48.099845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:48.099845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:48.099845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:48.099845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:48.099845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:48.099845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:48.099845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:48.099845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:48.099845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:48.099845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:48.099845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:48.099845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:48.099845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:48.100758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:48.100758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:48.100758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:48.100758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:48.100758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:48.100758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:48.100758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:48.100758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:48.100758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:48.100758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:48.100758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:48.100758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:48.100758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:48.100758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:48.100758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:48.100758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:48.101540 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:48.101540 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:48.101540 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:48.101540 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:48.101540 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:48.101540 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:48.101540 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:48.101540 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:48.101540 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:48.101540 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:48.101540 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:48.101540 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:48.107083 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 196/388] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Jun 27 18:26:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:48.113093 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9808a7-6364-447c-96ce-0f4f7003fa55 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:48.116475 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:48.176589 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd098662-2933-4e0d-88dc-059c18a8e61c None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:48.198414 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:48.238536 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9808a7-6364-447c-96ce-0f4f7003fa55 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:48.247861 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6bc81-90b2-437f-a600-589caa9d9339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:48.249037 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6bc81-90b2-437f-a600-589caa9d9339 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:48.249526 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6bc81-90b2-437f-a600-589caa9d9339 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:48.250012 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6bc81-90b2-437f-a600-589caa9d9339 None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:48.251571 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6bc81-90b2-437f-a600-589caa9d9339 None admin] RBAC: Authorizing `identity:get_project(project_id=d1f168a334d74b6bb18fa28c3f9774d1)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:48.253488 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6bc81-90b2-437f-a600-589caa9d9339 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:48.260028 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 193/389] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:48 2022] GET /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:48.266018 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6bc81-90b2-437f-a600-589caa9d9339 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:48.275224 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd611a6d-ba23-41f7-972b-45346eb519cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:48.275894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd611a6d-ba23-41f7-972b-45346eb519cc None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:48.276181 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd611a6d-ba23-41f7-972b-45346eb519cc None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:48.276466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd611a6d-ba23-41f7-972b-45346eb519cc None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:48.277223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd611a6d-ba23-41f7-972b-45346eb519cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:48.277630 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd611a6d-ba23-41f7-972b-45346eb519cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:48.279077 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd611a6d-ba23-41f7-972b-45346eb519cc None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:48.288031 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 197/390] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:26:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:48.293925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd611a6d-ba23-41f7-972b-45346eb519cc None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:48.403073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd611a6d-ba23-41f7-972b-45346eb519cc None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:48.430503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:48.440992 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1fddf-328a-4c2c-9358-ed85ea4061fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:48.449494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1fddf-328a-4c2c-9358-ed85ea4061fa None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:48.449494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1fddf-328a-4c2c-9358-ed85ea4061fa None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:48.449494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1fddf-328a-4c2c-9358-ed85ea4061fa None admin] PATH_INFO: `/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:48.449494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f1fddf-328a-4c2c-9358-ed85ea4061fa None admin] RBAC: Authorizing `identity:get_group(group_id=9f7e220e2ba14938ae0cac70c096cb8b)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:48.449494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f1fddf-328a-4c2c-9358-ed85ea4061fa None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:48.449494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 198/391] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Jun 27 18:26:48 2022] GET /identity/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:48.453608 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1fddf-328a-4c2c-9358-ed85ea4061fa None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:48.455679 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6bc81-90b2-437f-a600-589caa9d9339 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:48.504010 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:48.510088 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ffff1-f7d6-4fab-ac31-5f98f61fe3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:48.511021 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ffff1-f7d6-4fab-ac31-5f98f61fe3cf None admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:48.511299 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ffff1-f7d6-4fab-ac31-5f98f61fe3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:48.511571 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ffff1-f7d6-4fab-ac31-5f98f61fe3cf None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1/groups/9f7e220e2ba14938ae0cac70c096cb8b/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:48.513885 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ffff1-f7d6-4fab-ac31-5f98f61fe3cf None admin] RBAC: Authorizing `identity:create_grant(project_id=d1f168a334d74b6bb18fa28c3f9774d1, group_id=9f7e220e2ba14938ae0cac70c096cb8b, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:48.515340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ffff1-f7d6-4fab-ac31-5f98f61fe3cf None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:48.571633 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1fddf-328a-4c2c-9358-ed85ea4061fa None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:48.617903 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 194/392] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Jun 27 18:26:48 2022] PUT /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1/groups/9f7e220e2ba14938ae0cac70c096cb8b/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:26:48.619811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbe971-71b2-4f18-acbc-e6dd9d6f1613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:48.620691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbe971-71b2-4f18-acbc-e6dd9d6f1613 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:48.620967 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbe971-71b2-4f18-acbc-e6dd9d6f1613 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:48.621835 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbe971-71b2-4f18-acbc-e6dd9d6f1613 None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:48.627392 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bbe971-71b2-4f18-acbc-e6dd9d6f1613 None admin] RBAC: Authorizing `identity:get_project(project_id=d1f168a334d74b6bb18fa28c3f9774d1)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:48.629076 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bbe971-71b2-4f18-acbc-e6dd9d6f1613 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:48.631284 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 199/393] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:48 2022] GET /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:48.638031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ffff1-f7d6-4fab-ac31-5f98f61fe3cf None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:48.779716 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ffff1-f7d6-4fab-ac31-5f98f61fe3cf None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:48.827441 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a987c-316b-499a-b588-43f6f70aced5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:48.828333 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a987c-316b-499a-b588-43f6f70aced5 None admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:48.828617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a987c-316b-499a-b588-43f6f70aced5 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:48.828893 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a987c-316b-499a-b588-43f6f70aced5 None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1/groups/9f7e220e2ba14938ae0cac70c096cb8b/roles/44d7d6bfe7eb441bbc6a301a5b6784ad` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:48.832212 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a987c-316b-499a-b588-43f6f70aced5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1f168a334d74b6bb18fa28c3f9774d1, group_id=9f7e220e2ba14938ae0cac70c096cb8b, role_id=44d7d6bfe7eb441bbc6a301a5b6784ad)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:48.833659 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a987c-316b-499a-b588-43f6f70aced5 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:48.927195 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 195/394] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Jun 27 18:26:48 2022] PUT /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1/groups/9f7e220e2ba14938ae0cac70c096cb8b/roles/44d7d6bfe7eb441bbc6a301a5b6784ad => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:26:49.087320 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757e69a-aac5-432f-8037-bee5d8ec91b0 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:49.087845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757e69a-aac5-432f-8037-bee5d8ec91b0 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:49.087924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757e69a-aac5-432f-8037-bee5d8ec91b0 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:49.089518 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 200/395] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:49.097085 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c17ab-be8d-4d47-818d-f5c6a07d73ff None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:49.097385 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c17ab-be8d-4d47-818d-f5c6a07d73ff None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:49.097653 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c17ab-be8d-4d47-818d-f5c6a07d73ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:49.118256 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-c83c17ab-be8d-4d47-818d-f5c6a07d73ff None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:49.119539 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-c83c17ab-be8d-4d47-818d-f5c6a07d73ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:49.167044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:49.199646 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 196/396] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:49.232415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757e69a-aac5-432f-8037-bee5d8ec91b0 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:49.309398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:49.354365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757e69a-aac5-432f-8037-bee5d8ec91b0 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:49.397948 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870962ad-a081-47eb-9f2a-d57b49b8b5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:49.398855 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870962ad-a081-47eb-9f2a-d57b49b8b5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:49.399118 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870962ad-a081-47eb-9f2a-d57b49b8b5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:49.399392 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870962ad-a081-47eb-9f2a-d57b49b8b5c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:49.411854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870962ad-a081-47eb-9f2a-d57b49b8b5c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:49.422374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-870962ad-a081-47eb-9f2a-d57b49b8b5c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:49.422374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:49.422374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:49.422374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:49.422374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:49.422374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:49.422374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:49.422374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:49.422374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:49.422374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:49.422374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:49.422374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:49.422374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:49.422374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:49.422374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:49.422374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:49.423309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:49.423309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:49.423309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:49.423309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:49.423309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:49.423309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:49.423309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:49.423309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:49.423309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:49.423309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:49.423309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:49.423309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:49.423309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:49.423309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:49.423309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:49.423309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:49.424108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:49.424108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:49.424108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:49.424108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:49.424108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:49.424108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:49.424108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:49.424108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:49.424108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:49.424108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:49.424108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:49.424108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:49.424108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 201/397] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:49.429935 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c17ab-be8d-4d47-818d-f5c6a07d73ff None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:49.554421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c17ab-be8d-4d47-818d-f5c6a07d73ff None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:49.594964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce38cfc-a6a3-4dc5-ac69-8365493de3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:49.596002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce38cfc-a6a3-4dc5-ac69-8365493de3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:49.596511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce38cfc-a6a3-4dc5-ac69-8365493de3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:49.596990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce38cfc-a6a3-4dc5-ac69-8365493de3e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:49.597936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce38cfc-a6a3-4dc5-ac69-8365493de3e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:49.598475 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce38cfc-a6a3-4dc5-ac69-8365493de3e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:49.600074 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce38cfc-a6a3-4dc5-ac69-8365493de3e1 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:49.610596 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 197/398] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:49.615772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870962ad-a081-47eb-9f2a-d57b49b8b5c4 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:49.774228 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870962ad-a081-47eb-9f2a-d57b49b8b5c4 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:49.805297 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:49.817627 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337578dd-42ce-4d2e-860b-b09912473fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:49.818572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337578dd-42ce-4d2e-860b-b09912473fce None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:49.818860 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337578dd-42ce-4d2e-860b-b09912473fce None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:49.819137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337578dd-42ce-4d2e-860b-b09912473fce None admin] PATH_INFO: `/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:49.823062 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337578dd-42ce-4d2e-860b-b09912473fce None admin] RBAC: Authorizing `identity:get_group(group_id=9f7e220e2ba14938ae0cac70c096cb8b)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:49.825500 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337578dd-42ce-4d2e-860b-b09912473fce None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:49.827496 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 202/399] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Jun 27 18:26:49 2022] GET /identity/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:49.833010 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce38cfc-a6a3-4dc5-ac69-8365493de3e1 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:49.890151 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:49.928063 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afd3be-8f87-442a-bb06-53ae19562271 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:49.928370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afd3be-8f87-442a-bb06-53ae19562271 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:49.928626 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afd3be-8f87-442a-bb06-53ae19562271 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:49.930245 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 203/400] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:49.935475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d15059-b369-4e7a-8afb-825eda2da7c5 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:49.935765 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d15059-b369-4e7a-8afb-825eda2da7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:49.936046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d15059-b369-4e7a-8afb-825eda2da7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:49.959972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce38cfc-a6a3-4dc5-ac69-8365493de3e1 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:49.975427 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-02d15059-b369-4e7a-8afb-825eda2da7c5 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:49.976623 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-02d15059-b369-4e7a-8afb-825eda2da7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:49.998635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad4dc0-4d11-4b39-a5d5-374d534d7a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:49.999490 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad4dc0-4d11-4b39-a5d5-374d534d7a36 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:49.999766 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad4dc0-4d11-4b39-a5d5-374d534d7a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:50.000035 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad4dc0-4d11-4b39-a5d5-374d534d7a36 None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:50.001338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ad4dc0-4d11-4b39-a5d5-374d534d7a36 None admin] RBAC: Authorizing `identity:get_project(project_id=4d7f983bf3f84850a4a59357572371d9)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:50.002802 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ad4dc0-4d11-4b39-a5d5-374d534d7a36 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:50.004961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 198/401] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:49 2022] GET /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:50.013719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad4dc0-4d11-4b39-a5d5-374d534d7a36 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:50.125097 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad4dc0-4d11-4b39-a5d5-374d534d7a36 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:50.154532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:50.163595 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db25ba-f274-422e-8b5e-db0d7fcc188d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:50.164061 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 204/402] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:50.164556 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db25ba-f274-422e-8b5e-db0d7fcc188d None admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:50.164945 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db25ba-f274-422e-8b5e-db0d7fcc188d None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:50.165292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db25ba-f274-422e-8b5e-db0d7fcc188d None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9/groups/9f7e220e2ba14938ae0cac70c096cb8b/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:50.167389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db25ba-f274-422e-8b5e-db0d7fcc188d None admin] RBAC: Authorizing `identity:create_grant(project_id=4d7f983bf3f84850a4a59357572371d9, group_id=9f7e220e2ba14938ae0cac70c096cb8b, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:50.169007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db25ba-f274-422e-8b5e-db0d7fcc188d None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:50.188778 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d15059-b369-4e7a-8afb-825eda2da7c5 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:50.277246 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 199/403] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Jun 27 18:26:50 2022] PUT /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9/groups/9f7e220e2ba14938ae0cac70c096cb8b/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:26:50.322282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d15059-b369-4e7a-8afb-825eda2da7c5 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:50.354744 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:50.373230 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd96cb-8869-4b7d-9bac-dfe13ef81fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:50.374048 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd96cb-8869-4b7d-9bac-dfe13ef81fea None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:50.374331 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd96cb-8869-4b7d-9bac-dfe13ef81fea None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:50.374629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd96cb-8869-4b7d-9bac-dfe13ef81fea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:50.389552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fd96cb-8869-4b7d-9bac-dfe13ef81fea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:50.401013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-61fd96cb-8869-4b7d-9bac-dfe13ef81fea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:50.401013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:50.401013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:50.401013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:50.401013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:50.401013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:50.401013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:50.401013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:50.401013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:50.401013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:50.401013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:50.401013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:50.401013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:50.401013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:50.401013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:50.401013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:50.401961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:50.401961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:50.401961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:50.401961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:50.401961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:50.401961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:50.401961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:50.401961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:50.401961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:50.401961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:50.401961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:50.401961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:50.401961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:50.401961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:50.401961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:50.401961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:50.402812 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:50.402812 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:50.402812 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:50.402812 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:50.402812 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:50.402812 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:50.402812 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:50.402812 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:50.402812 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:50.402812 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:50.402812 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:50.402812 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:50.402812 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 205/404] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Jun 27 18:26:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:26:50.408476 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db25ba-f274-422e-8b5e-db0d7fcc188d None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:50.437590 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604d175-e720-4b17-b096-6615e224fdec None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:50.437905 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604d175-e720-4b17-b096-6615e224fdec None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:50.439133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604d175-e720-4b17-b096-6615e224fdec None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:50.439777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 206/405] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:26:50.447878 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83736c71-4741-4587-bd3f-274306a2c3ad None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:50.448182 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83736c71-4741-4587-bd3f-274306a2c3ad None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:50.448436 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83736c71-4741-4587-bd3f-274306a2c3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:50.488109 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-83736c71-4741-4587-bd3f-274306a2c3ad None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:50.489354 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-83736c71-4741-4587-bd3f-274306a2c3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:50.564059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db25ba-f274-422e-8b5e-db0d7fcc188d None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:50.578878 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 207/406] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:26:50.601753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83736c71-4741-4587-bd3f-274306a2c3ad None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:50.604045 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd0c8f-3b4f-4b35-99bf-14d2c7a77c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:50.604909 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd0c8f-3b4f-4b35-99bf-14d2c7a77c82 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:50.605330 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd0c8f-3b4f-4b35-99bf-14d2c7a77c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:50.606557 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd0c8f-3b4f-4b35-99bf-14d2c7a77c82 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:50.607728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd0c8f-3b4f-4b35-99bf-14d2c7a77c82 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:50.608200 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd0c8f-3b4f-4b35-99bf-14d2c7a77c82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:50.609735 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd0c8f-3b4f-4b35-99bf-14d2c7a77c82 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:50.628979 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 200/407] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:26:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:50.634747 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd0c8f-3b4f-4b35-99bf-14d2c7a77c82 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:50.670826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:50.772743 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd0c8f-3b4f-4b35-99bf-14d2c7a77c82 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:50.784899 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83736c71-4741-4587-bd3f-274306a2c3ad None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:50.797503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:50.822024 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78db82-b431-464d-a045-7be3df167466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:50.822910 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78db82-b431-464d-a045-7be3df167466 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:50.823195 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78db82-b431-464d-a045-7be3df167466 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:50.823466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78db82-b431-464d-a045-7be3df167466 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:50.833934 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78db82-b431-464d-a045-7be3df167466 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:50.842504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-8a78db82-b431-464d-a045-7be3df167466 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:26:50.842504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:50.842504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:50.842504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:50.842504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:50.842504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:50.842504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:50.842504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:50.842504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:50.842504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:50.842504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:50.842504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:50.842504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:50.842504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:50.842504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:50.842504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:50.843416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:50.843416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:50.843416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:50.843416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:50.843416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:50.843416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:50.843416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:50.843416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:50.843416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:50.843416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:50.843416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:50.843416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:50.843416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:50.843416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:50.843416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:50.843416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:50.844215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:50.844215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:50.844215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:50.844215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:50.844215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:50.844215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:50.844215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:50.844215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:50.844215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:50.844215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:50.844215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:26:50.844215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:50.844215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 208/408] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:26:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:26:50.851612 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78db82-b431-464d-a045-7be3df167466 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:50.875664 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcea0b-381c-435f-bcdd-bd96f3a38cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:50.876543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcea0b-381c-435f-bcdd-bd96f3a38cec None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:50.876828 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcea0b-381c-435f-bcdd-bd96f3a38cec None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:50.877105 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcea0b-381c-435f-bcdd-bd96f3a38cec None admin] PATH_INFO: `/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:50.883130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcea0b-381c-435f-bcdd-bd96f3a38cec None admin] RBAC: Authorizing `identity:get_group(group_id=9f7e220e2ba14938ae0cac70c096cb8b)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:50.884607 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcea0b-381c-435f-bcdd-bd96f3a38cec None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:50.886601 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 201/409] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Jun 27 18:26:50 2022] GET /identity/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:50.905481 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcea0b-381c-435f-bcdd-bd96f3a38cec None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:50.967703 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78db82-b431-464d-a045-7be3df167466 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:51.004345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8756c-195b-4467-9460-156f044121eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:51.005008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8756c-195b-4467-9460-156f044121eb None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:51.005292 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8756c-195b-4467-9460-156f044121eb None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:51.005571 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8756c-195b-4467-9460-156f044121eb None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:51.006323 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e8756c-195b-4467-9460-156f044121eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:51.006660 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e8756c-195b-4467-9460-156f044121eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:51.008578 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e8756c-195b-4467-9460-156f044121eb None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:51.018242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 209/410] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:51.023495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8756c-195b-4467-9460-156f044121eb None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:51.075126 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:51.146116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8756c-195b-4467-9460-156f044121eb None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:51.174723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:51.189428 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcea0b-381c-435f-bcdd-bd96f3a38cec None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:51.206663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee270552-dff5-4871-a858-e41849529071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:51.208177 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee270552-dff5-4871-a858-e41849529071 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:51.208678 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee270552-dff5-4871-a858-e41849529071 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:51.210723 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee270552-dff5-4871-a858-e41849529071 None admin] PATH_INFO: `/v3/groups/6e27a65d60a144a88caed67204d4a9bb` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:51.212327 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee270552-dff5-4871-a858-e41849529071 None admin] RBAC: Authorizing `identity:get_group(group_id=6e27a65d60a144a88caed67204d4a9bb)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:51.213987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee270552-dff5-4871-a858-e41849529071 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:51.218094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 210/411] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Jun 27 18:26:51 2022] GET /identity/v3/groups/6e27a65d60a144a88caed67204d4a9bb => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:51.225529 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee270552-dff5-4871-a858-e41849529071 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:51.266895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63805606-678f-4170-ab02-5f53d61e6dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:51.268150 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63805606-678f-4170-ab02-5f53d61e6dab None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:51.268632 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63805606-678f-4170-ab02-5f53d61e6dab None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:51.269132 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63805606-678f-4170-ab02-5f53d61e6dab None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:51.270705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63805606-678f-4170-ab02-5f53d61e6dab None admin] RBAC: Authorizing `identity:get_project(project_id=4d7f983bf3f84850a4a59357572371d9)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:51.272420 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63805606-678f-4170-ab02-5f53d61e6dab None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:51.279224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 202/412] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:50 2022] GET /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9 => generated 302 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:51.282423 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63805606-678f-4170-ab02-5f53d61e6dab None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:51.411652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63805606-678f-4170-ab02-5f53d61e6dab None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:51.447971 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:51.451865 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee270552-dff5-4871-a858-e41849529071 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:51.452728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe5cadb-0093-45c7-bc04-5fbfcab7c19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:51.453697 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5cadb-0093-45c7-bc04-5fbfcab7c19d None admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:51.454168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5cadb-0093-45c7-bc04-5fbfcab7c19d None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:51.454533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5cadb-0093-45c7-bc04-5fbfcab7c19d None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9/groups/9f7e220e2ba14938ae0cac70c096cb8b/roles/44d7d6bfe7eb441bbc6a301a5b6784ad` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:51.457725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe5cadb-0093-45c7-bc04-5fbfcab7c19d None admin] RBAC: Authorizing `identity:create_grant(project_id=4d7f983bf3f84850a4a59357572371d9, group_id=9f7e220e2ba14938ae0cac70c096cb8b, role_id=44d7d6bfe7eb441bbc6a301a5b6784ad)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:51.459288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe5cadb-0093-45c7-bc04-5fbfcab7c19d None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:51.491597 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d9951-9fc4-4f3c-bd0d-dbf20d9ab8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:51.492713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d9951-9fc4-4f3c-bd0d-dbf20d9ab8e2 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:51.493182 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d9951-9fc4-4f3c-bd0d-dbf20d9ab8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:51.493643 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d9951-9fc4-4f3c-bd0d-dbf20d9ab8e2 None admin] PATH_INFO: `/v3/projects/d9ffe361a6094e199bb35c633199d02f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:51.495021 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d9951-9fc4-4f3c-bd0d-dbf20d9ab8e2 None admin] RBAC: Authorizing `identity:get_project(project_id=d9ffe361a6094e199bb35c633199d02f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:51.497507 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d9951-9fc4-4f3c-bd0d-dbf20d9ab8e2 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:51.499758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 211/413] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:51 2022] GET /identity/v3/projects/d9ffe361a6094e199bb35c633199d02f => generated 344 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:51.500749 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebe8d5-1714-4e8e-9919-9392c8212f15 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:51.501039 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebe8d5-1714-4e8e-9919-9392c8212f15 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:51.501315 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebe8d5-1714-4e8e-9919-9392c8212f15 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:51.502799 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 212/414] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:51.507400 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebe8d5-1714-4e8e-9919-9392c8212f15 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:51.522409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 203/415] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Jun 27 18:26:51 2022] PUT /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9/groups/9f7e220e2ba14938ae0cac70c096cb8b/roles/44d7d6bfe7eb441bbc6a301a5b6784ad => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:26:51.522409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f6d62-334a-4f9e-aac4-3f811f6def9f None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:51.522409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f6d62-334a-4f9e-aac4-3f811f6def9f None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:51.522409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f6d62-334a-4f9e-aac4-3f811f6def9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:51.562430 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-fa9f6d62-334a-4f9e-aac4-3f811f6def9f None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:51.565837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-fa9f6d62-334a-4f9e-aac4-3f811f6def9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:51.608805 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:51.718401 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 204/416] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:51.721040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebe8d5-1714-4e8e-9919-9392c8212f15 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:51.741960 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f6d62-334a-4f9e-aac4-3f811f6def9f None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:51.769513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e8042-274e-48b8-822d-7ddd3861b10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:51.771342 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e8042-274e-48b8-822d-7ddd3861b10a None admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:51.772088 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e8042-274e-48b8-822d-7ddd3861b10a None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:51.772647 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e8042-274e-48b8-822d-7ddd3861b10a None admin] PATH_INFO: `/v3/projects/d9ffe361a6094e199bb35c633199d02f/groups/6e27a65d60a144a88caed67204d4a9bb/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:51.777042 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e8042-274e-48b8-822d-7ddd3861b10a None admin] RBAC: Authorizing `identity:create_grant(project_id=d9ffe361a6094e199bb35c633199d02f, group_id=6e27a65d60a144a88caed67204d4a9bb, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:51.780297 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e8042-274e-48b8-822d-7ddd3861b10a None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:51.858949 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 213/417] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Jun 27 18:26:51 2022] PUT /identity/v3/projects/d9ffe361a6094e199bb35c633199d02f/groups/6e27a65d60a144a88caed67204d4a9bb/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:26:51.877101 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f6d62-334a-4f9e-aac4-3f811f6def9f None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:51.896304 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb4290-c6de-40d9-96c7-d6d8015cfbe9 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:51.896642 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb4290-c6de-40d9-96c7-d6d8015cfbe9 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:51.896870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb4290-c6de-40d9-96c7-d6d8015cfbe9 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:51.902729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 214/418] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:26:51.904206 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c4ce1-0a6e-4bde-8342-0b76c17a0614 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:51.904415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c4ce1-0a6e-4bde-8342-0b76c17a0614 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:51.904698 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c4ce1-0a6e-4bde-8342-0b76c17a0614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:51.930685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeca81f-66dd-43b3-9582-42b8ad0013ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:51.931516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeca81f-66dd-43b3-9582-42b8ad0013ee None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:51.931797 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeca81f-66dd-43b3-9582-42b8ad0013ee None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:51.932071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeca81f-66dd-43b3-9582-42b8ad0013ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:51.933871 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-267c4ce1-0a6e-4bde-8342-0b76c17a0614 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:51.935141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-267c4ce1-0a6e-4bde-8342-0b76c17a0614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:51.950482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeca81f-66dd-43b3-9582-42b8ad0013ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:51.958713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-efeca81f-66dd-43b3-9582-42b8ad0013ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:51.958713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:51.958713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:51.958713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:51.958713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:51.958713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:51.958713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:51.958713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:51.958713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:51.958713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:51.958713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:51.958713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:51.958713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:51.958713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:51.958713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:51.958713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:51.959656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:51.959656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:51.959656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:51.959656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:51.959656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:51.959656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:51.959656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:51.959656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:51.959656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:51.959656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:51.959656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:51.959656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:51.959656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:51.959656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:51.959656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:51.959656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:51.960688 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:51.960688 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:51.960688 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:51.960688 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:51.960688 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:51.960688 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:51.960688 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:51.960688 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:51.960688 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:51.960688 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:51.960688 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:51.960688 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:26:51.960688 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 205/419] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:26:51.975577 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeca81f-66dd-43b3-9582-42b8ad0013ee None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:51.982512 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:52.007622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:52.060869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 215/420] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:52.089770 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c4ce1-0a6e-4bde-8342-0b76c17a0614 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:52.135101 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeca81f-66dd-43b3-9582-42b8ad0013ee None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:52.192271 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df26a06-dbfb-48ea-bf65-f5fe88976b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:52.192948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df26a06-dbfb-48ea-bf65-f5fe88976b06 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:52.193231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df26a06-dbfb-48ea-bf65-f5fe88976b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:52.193506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df26a06-dbfb-48ea-bf65-f5fe88976b06 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:52.194260 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df26a06-dbfb-48ea-bf65-f5fe88976b06 None admin] RBAC: Adding query filter params (name=member) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:52.194875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df26a06-dbfb-48ea-bf65-f5fe88976b06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:52.197006 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df26a06-dbfb-48ea-bf65-f5fe88976b06 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:52.215363 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 206/421] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:52.221164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df26a06-dbfb-48ea-bf65-f5fe88976b06 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:52.235977 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c4ce1-0a6e-4bde-8342-0b76c17a0614 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:52.313174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadf211-272b-4424-a93b-def8d522449d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:52.315019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadf211-272b-4424-a93b-def8d522449d None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:52.315310 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadf211-272b-4424-a93b-def8d522449d None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:52.315594 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadf211-272b-4424-a93b-def8d522449d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:52.339217 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadf211-272b-4424-a93b-def8d522449d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:52.350984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-daadf211-272b-4424-a93b-def8d522449d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:52.350984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:52.350984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:52.350984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:52.350984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:52.350984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:52.350984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:52.350984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:52.350984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:52.350984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:52.350984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:52.350984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:52.350984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:52.350984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:52.350984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:52.350984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:52.351902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:52.351902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:52.351902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:52.351902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:52.351902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:52.351902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:52.351902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:52.351902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:52.351902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:52.351902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:52.351902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:52.351902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:52.351902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:52.351902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:52.351902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:52.351902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:52.352685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:52.352685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:52.352685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:52.352685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:52.352685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:52.352685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:52.352685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:52.352685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:52.352685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:52.352685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:52.352685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:52.352685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:52.358663 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:52.361345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 216/422] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Jun 27 18:26:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:26:52.370433 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadf211-272b-4424-a93b-def8d522449d None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:52.393709 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:52.403197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df26a06-dbfb-48ea-bf65-f5fe88976b06 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:52.482836 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadf211-272b-4424-a93b-def8d522449d None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:52.504094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464a993-5698-496e-b282-95bd06c8edfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:52.504750 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464a993-5698-496e-b282-95bd06c8edfb None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:52.505035 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464a993-5698-496e-b282-95bd06c8edfb None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:52.505312 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464a993-5698-496e-b282-95bd06c8edfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:52.520896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607749dc-32a3-4ffa-b646-5a4db9966780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:52.521820 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607749dc-32a3-4ffa-b646-5a4db9966780 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:52.522245 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607749dc-32a3-4ffa-b646-5a4db9966780 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:52.522693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607749dc-32a3-4ffa-b646-5a4db9966780 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:52.523658 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607749dc-32a3-4ffa-b646-5a4db9966780 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:52.524111 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607749dc-32a3-4ffa-b646-5a4db9966780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:52.525704 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607749dc-32a3-4ffa-b646-5a4db9966780 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:52.535600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 217/423] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:26:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:52.547917 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607749dc-32a3-4ffa-b646-5a4db9966780 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:52.613710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d464a993-5698-496e-b282-95bd06c8edfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:52.615188 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d464a993-5698-496e-b282-95bd06c8edfb None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:52.636430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:52.657110 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607749dc-32a3-4ffa-b646-5a4db9966780 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:52.693544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84217cf-ee9e-433d-9dc3-ff66c6da7f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:52.694623 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84217cf-ee9e-433d-9dc3-ff66c6da7f07 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:52.695051 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84217cf-ee9e-433d-9dc3-ff66c6da7f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:52.695471 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84217cf-ee9e-433d-9dc3-ff66c6da7f07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:52.734758 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:52.771747 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 207/424] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:26:52.774029 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67923a4a-95ca-4ba9-8b14-7a419ac77669 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:52.774567 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67923a4a-95ca-4ba9-8b14-7a419ac77669 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:52.775338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67923a4a-95ca-4ba9-8b14-7a419ac77669 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:52.775916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84217cf-ee9e-433d-9dc3-ff66c6da7f07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:52.777144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 208/425] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:52.777497 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84217cf-ee9e-433d-9dc3-ff66c6da7f07 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:52.778405 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67923a4a-95ca-4ba9-8b14-7a419ac77669 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:52.870983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:52.892145 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 218/426] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:26:52.893490 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de60cd4-fc4a-42fe-b45c-0daa62d65197 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:52.893785 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de60cd4-fc4a-42fe-b45c-0daa62d65197 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:52.894067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de60cd4-fc4a-42fe-b45c-0daa62d65197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:52.898998 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67923a4a-95ca-4ba9-8b14-7a419ac77669 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:52.911060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-5de60cd4-fc4a-42fe-b45c-0daa62d65197 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:52.912258 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-5de60cd4-fc4a-42fe-b45c-0daa62d65197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:52.938829 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e463da5-6564-4383-88ab-b1510e012ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:52.940105 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e463da5-6564-4383-88ab-b1510e012ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:52.940670 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e463da5-6564-4383-88ab-b1510e012ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:52.941231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e463da5-6564-4383-88ab-b1510e012ae2 None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:52.942659 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e463da5-6564-4383-88ab-b1510e012ae2 None admin] RBAC: Authorizing `identity:get_project(project_id=d1f168a334d74b6bb18fa28c3f9774d1)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:52.944499 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e463da5-6564-4383-88ab-b1510e012ae2 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:52.947060 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 209/427] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:52 2022] GET /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:52.948064 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e463da5-6564-4383-88ab-b1510e012ae2 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:52.983029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 219/428] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:52.983941 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de60cd4-fc4a-42fe-b45c-0daa62d65197 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:52.996951 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:53.058367 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e463da5-6564-4383-88ab-b1510e012ae2 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:53.091266 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de60cd4-fc4a-42fe-b45c-0daa62d65197 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:53.097390 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b9c7a-c497-4788-9da8-fd72c3abf2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:53.098535 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b9c7a-c497-4788-9da8-fd72c3abf2f1 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:53.099127 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b9c7a-c497-4788-9da8-fd72c3abf2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:53.099613 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b9c7a-c497-4788-9da8-fd72c3abf2f1 None admin] PATH_INFO: `/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:53.101000 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b9c7a-c497-4788-9da8-fd72c3abf2f1 None admin] RBAC: Authorizing `identity:get_project(project_id=d1f168a334d74b6bb18fa28c3f9774d1)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:53.102854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:53.103542 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b9c7a-c497-4788-9da8-fd72c3abf2f1 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:53.105943 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 210/429] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:52 2022] GET /identity/v3/projects/d1f168a334d74b6bb18fa28c3f9774d1 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:53.107274 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b9c7a-c497-4788-9da8-fd72c3abf2f1 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:53.129588 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4c131-e51b-4ca0-8e93-25ab3d5c15ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:53.130360 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4c131-e51b-4ca0-8e93-25ab3d5c15ab None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:53.130668 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4c131-e51b-4ca0-8e93-25ab3d5c15ab None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:53.131000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4c131-e51b-4ca0-8e93-25ab3d5c15ab None admin] PATH_INFO: `/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:53.132353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4c131-e51b-4ca0-8e93-25ab3d5c15ab None admin] RBAC: Authorizing `identity:get_group(group_id=9f7e220e2ba14938ae0cac70c096cb8b)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:53.133811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4c131-e51b-4ca0-8e93-25ab3d5c15ab None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:53.135951 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 220/430] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Jun 27 18:26:52 2022] GET /identity/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:53.137746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4c131-e51b-4ca0-8e93-25ab3d5c15ab None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:53.213987 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b9c7a-c497-4788-9da8-fd72c3abf2f1 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:53.230937 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:53.247191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4c131-e51b-4ca0-8e93-25ab3d5c15ab None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:53.252872 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb035a4-68ee-4a09-a4bc-86e07e2d607b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:53.254027 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb035a4-68ee-4a09-a4bc-86e07e2d607b None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:53.254514 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb035a4-68ee-4a09-a4bc-86e07e2d607b None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:53.254960 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb035a4-68ee-4a09-a4bc-86e07e2d607b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:53.275634 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb035a4-68ee-4a09-a4bc-86e07e2d607b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:53.278608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-1cb035a4-68ee-4a09-a4bc-86e07e2d607b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:53.278608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:53.278608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:53.278608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:53.278608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:53.278608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:53.278608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:53.278608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:53.278608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:53.278608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:53.278608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:53.278608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:53.278608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:53.278608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:53.278608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:53.278608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:53.279511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:53.279511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:53.279511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:53.279511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:53.279511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:53.279511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:53.279511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:53.279511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:53.279511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:53.279511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:53.279511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:53.279511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:53.279511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:53.279511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:53.279511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:53.279511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:53.280322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:53.280322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:53.280322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:53.280322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:53.280322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:53.280322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:53.280322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:53.280322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:53.280322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:53.280322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:53.280322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 18:26:53.280322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:26:53.283174 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 211/431] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:53.284308 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb035a4-68ee-4a09-a4bc-86e07e2d607b None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:53.287029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cea74-deb9-427f-b945-ee8720b62609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:53.287973 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cea74-deb9-427f-b945-ee8720b62609 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:53.288333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cea74-deb9-427f-b945-ee8720b62609 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:53.288685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cea74-deb9-427f-b945-ee8720b62609 None admin] PATH_INFO: `/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:53.290839 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934cea74-deb9-427f-b945-ee8720b62609 None admin] RBAC: Authorizing `identity:get_group(group_id=9f7e220e2ba14938ae0cac70c096cb8b)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:53.292465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934cea74-deb9-427f-b945-ee8720b62609 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:53.294660 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 221/432] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Jun 27 18:26:53 2022] GET /identity/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:53.295386 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cea74-deb9-427f-b945-ee8720b62609 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:53.345308 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:53.401947 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb035a4-68ee-4a09-a4bc-86e07e2d607b None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:53.415431 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cea74-deb9-427f-b945-ee8720b62609 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:53.448968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77657f-052c-4b2f-82d9-bbeb3c66f911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:53.450281 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77657f-052c-4b2f-82d9-bbeb3c66f911 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:53.450846 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77657f-052c-4b2f-82d9-bbeb3c66f911 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:53.451348 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77657f-052c-4b2f-82d9-bbeb3c66f911 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:53.452409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa77657f-052c-4b2f-82d9-bbeb3c66f911 None admin] RBAC: Adding query filter params (group.id=9f7e220e2ba14938ae0cac70c096cb8b, role.id=b45aa1c36d974fe79c809afbfcd01945, scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:53.452992 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa77657f-052c-4b2f-82d9-bbeb3c66f911 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:53.453610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d9f1a-ab44-48ef-b194-8a70383e6fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:53.454358 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d9f1a-ab44-48ef-b194-8a70383e6fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:53.454744 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d9f1a-ab44-48ef-b194-8a70383e6fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:53.455094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d9f1a-ab44-48ef-b194-8a70383e6fb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:53.455445 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa77657f-052c-4b2f-82d9-bbeb3c66f911 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:53.455977 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d9f1a-ab44-48ef-b194-8a70383e6fb8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:53.456352 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d9f1a-ab44-48ef-b194-8a70383e6fb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:53.457877 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d9f1a-ab44-48ef-b194-8a70383e6fb8 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:53.463959 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 212/433] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Jun 27 18:26:53 2022] GET /identity/v3/role_assignments?group.id=9f7e220e2ba14938ae0cac70c096cb8b&scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1&role.id=b45aa1c36d974fe79c809afbfcd01945 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:53.465085 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77657f-052c-4b2f-82d9-bbeb3c66f911 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:53.467917 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 222/434] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:26:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:53.478804 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d9f1a-ab44-48ef-b194-8a70383e6fb8 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:53.508176 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:53.592755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77657f-052c-4b2f-82d9-bbeb3c66f911 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:53.634476 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a66c2-9264-4042-aa22-6d5223484efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:53.635511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a66c2-9264-4042-aa22-6d5223484efa None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:53.636480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a66c2-9264-4042-aa22-6d5223484efa None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:53.637215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a66c2-9264-4042-aa22-6d5223484efa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:53.638434 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a66c2-9264-4042-aa22-6d5223484efa None admin] RBAC: Adding query filter params (group.id=9f7e220e2ba14938ae0cac70c096cb8b, role.id=44d7d6bfe7eb441bbc6a301a5b6784ad, scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:53.639546 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a66c2-9264-4042-aa22-6d5223484efa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:53.641497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a66c2-9264-4042-aa22-6d5223484efa None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:53.650176 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 213/435] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Jun 27 18:26:53 2022] GET /identity/v3/role_assignments?group.id=9f7e220e2ba14938ae0cac70c096cb8b&scope.project.id=d1f168a334d74b6bb18fa28c3f9774d1&role.id=44d7d6bfe7eb441bbc6a301a5b6784ad => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:53.697536 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d9f1a-ab44-48ef-b194-8a70383e6fb8 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:53.722782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:53.741398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775be9b-4c72-42e5-a688-6f9b088dc684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:53.747121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775be9b-4c72-42e5-a688-6f9b088dc684 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:53.747121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775be9b-4c72-42e5-a688-6f9b088dc684 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:53.747121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775be9b-4c72-42e5-a688-6f9b088dc684 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:53.821764 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775be9b-4c72-42e5-a688-6f9b088dc684 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:53.823286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775be9b-4c72-42e5-a688-6f9b088dc684 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:53.870514 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1474bc-0b4d-4239-8b34-8518a94e2725 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:53.871177 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1474bc-0b4d-4239-8b34-8518a94e2725 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:53.871464 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1474bc-0b4d-4239-8b34-8518a94e2725 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:53.873424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 214/436] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:53.878792 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335c0fa-7e89-46da-b517-1ae9203f5cfd None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:53.879075 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335c0fa-7e89-46da-b517-1ae9203f5cfd None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:53.879727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335c0fa-7e89-46da-b517-1ae9203f5cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:53.909395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-3335c0fa-7e89-46da-b517-1ae9203f5cfd None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:53.910630 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-3335c0fa-7e89-46da-b517-1ae9203f5cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:53.946445 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 223/437] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:26:53.951705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775be9b-4c72-42e5-a688-6f9b088dc684 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:53.963931 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:53.998710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 215/438] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:54.022519 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335c0fa-7e89-46da-b517-1ae9203f5cfd None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:54.059547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:54.076274 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775be9b-4c72-42e5-a688-6f9b088dc684 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:54.120758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44491aaf-ef0c-4e76-9952-7b4bcc91a6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:54.121968 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44491aaf-ef0c-4e76-9952-7b4bcc91a6bc None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:54.122254 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44491aaf-ef0c-4e76-9952-7b4bcc91a6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:54.122571 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44491aaf-ef0c-4e76-9952-7b4bcc91a6bc None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:54.123934 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44491aaf-ef0c-4e76-9952-7b4bcc91a6bc None admin] RBAC: Authorizing `identity:get_project(project_id=4d7f983bf3f84850a4a59357572371d9)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:54.125381 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44491aaf-ef0c-4e76-9952-7b4bcc91a6bc None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:54.127302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 224/439] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:53 2022] GET /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:54.133031 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44491aaf-ef0c-4e76-9952-7b4bcc91a6bc None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:54.154592 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335c0fa-7e89-46da-b517-1ae9203f5cfd None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:54.197781 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4ba45-130f-48cf-8df1-5b89ee77e706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:54.198739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4ba45-130f-48cf-8df1-5b89ee77e706 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:54.199025 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4ba45-130f-48cf-8df1-5b89ee77e706 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:54.199861 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4ba45-130f-48cf-8df1-5b89ee77e706 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:54.211737 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4ba45-130f-48cf-8df1-5b89ee77e706 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:54.220169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-85b4ba45-130f-48cf-8df1-5b89ee77e706 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:54.220169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:54.220169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:54.220169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:54.220169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:54.220169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:54.220169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:54.220169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:54.220169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:54.220169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:54.220169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:54.220169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:54.220169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:54.220169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:54.220169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:54.220169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:54.221066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:54.221066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:54.221066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:54.221066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:54.221066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:54.221066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:54.221066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:54.221066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:54.221066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:54.221066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:54.221066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:54.221066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:54.221066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:54.221066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:54.221066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:54.221066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:54.221870 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:54.221870 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:54.221870 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:54.221870 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:54.221870 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:54.221870 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:54.221870 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:54.221870 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:54.221870 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:54.221870 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:54.221870 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:26:54.221870 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:26:54.221870 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 216/440] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Jun 27 18:26:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:26:54.223627 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:54.228821 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4ba45-130f-48cf-8df1-5b89ee77e706 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:54.252785 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44491aaf-ef0c-4e76-9952-7b4bcc91a6bc None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:54.294486 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166fd59-1003-48f7-89ee-a615abaa4b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:54.295599 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166fd59-1003-48f7-89ee-a615abaa4b08 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:54.295880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166fd59-1003-48f7-89ee-a615abaa4b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:54.296155 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166fd59-1003-48f7-89ee-a615abaa4b08 None admin] PATH_INFO: `/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:54.297361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166fd59-1003-48f7-89ee-a615abaa4b08 None admin] RBAC: Authorizing `identity:get_group(group_id=9f7e220e2ba14938ae0cac70c096cb8b)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:54.298906 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166fd59-1003-48f7-89ee-a615abaa4b08 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:54.300539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 225/441] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Jun 27 18:26:54 2022] GET /identity/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:54.301912 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb500fb-e8f3-4864-a657-7b5ad86b330d None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:54.303367 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb500fb-e8f3-4864-a657-7b5ad86b330d None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:54.303367 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb500fb-e8f3-4864-a657-7b5ad86b330d None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:54.304523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 226/442] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:54.309594 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb500fb-e8f3-4864-a657-7b5ad86b330d None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:54.346108 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:54.358830 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4ba45-130f-48cf-8df1-5b89ee77e706 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:54.394266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec32351-a1ee-4180-accb-4040a76e66dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:54.394954 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec32351-a1ee-4180-accb-4040a76e66dd None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:54.395260 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec32351-a1ee-4180-accb-4040a76e66dd None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:54.395519 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec32351-a1ee-4180-accb-4040a76e66dd None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:54.396256 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec32351-a1ee-4180-accb-4040a76e66dd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:54.396557 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec32351-a1ee-4180-accb-4040a76e66dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:54.398025 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec32351-a1ee-4180-accb-4040a76e66dd None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:54.406560 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 217/443] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:26:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:54.407832 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240493f5-1728-4501-a6e6-0d6e526d149d None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:54.408118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240493f5-1728-4501-a6e6-0d6e526d149d None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:54.408404 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240493f5-1728-4501-a6e6-0d6e526d149d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:54.415495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb500fb-e8f3-4864-a657-7b5ad86b330d None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:54.425306 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-240493f5-1728-4501-a6e6-0d6e526d149d None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:54.426531 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-240493f5-1728-4501-a6e6-0d6e526d149d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:54.449858 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55abe8-9a88-4fd7-917a-04cbbf5cb115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:54.450534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55abe8-9a88-4fd7-917a-04cbbf5cb115 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:54.450820 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55abe8-9a88-4fd7-917a-04cbbf5cb115 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:54.451098 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55abe8-9a88-4fd7-917a-04cbbf5cb115 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:54.451949 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c55abe8-9a88-4fd7-917a-04cbbf5cb115 None admin] RBAC: Adding query filter params (group.id=9f7e220e2ba14938ae0cac70c096cb8b, role.id=b45aa1c36d974fe79c809afbfcd01945, scope.project.id=4d7f983bf3f84850a4a59357572371d9) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:54.452243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c55abe8-9a88-4fd7-917a-04cbbf5cb115 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:54.453691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c55abe8-9a88-4fd7-917a-04cbbf5cb115 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:54.459414 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 227/444] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Jun 27 18:26:54 2022] GET /identity/v3/role_assignments?group.id=9f7e220e2ba14938ae0cac70c096cb8b&scope.project.id=4d7f983bf3f84850a4a59357572371d9&role.id=b45aa1c36d974fe79c809afbfcd01945 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:54.460336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55abe8-9a88-4fd7-917a-04cbbf5cb115 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:54.472151 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:54.499955 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 218/445] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:54.525097 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240493f5-1728-4501-a6e6-0d6e526d149d None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:54.577839 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55abe8-9a88-4fd7-917a-04cbbf5cb115 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:54.618186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c08830-feb7-4ae0-81af-f7b93adc1b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:54.618885 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c08830-feb7-4ae0-81af-f7b93adc1b06 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:54.619181 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c08830-feb7-4ae0-81af-f7b93adc1b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:54.619435 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c08830-feb7-4ae0-81af-f7b93adc1b06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:54.620159 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:54.642342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240493f5-1728-4501-a6e6-0d6e526d149d None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:54.681160 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd606434-8988-4547-8c28-f06be2f28ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:54.682325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd606434-8988-4547-8c28-f06be2f28ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:54.682780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd606434-8988-4547-8c28-f06be2f28ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:54.683089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd606434-8988-4547-8c28-f06be2f28ed7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:54.695672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd606434-8988-4547-8c28-f06be2f28ed7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:26:54.701960 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c08830-feb7-4ae0-81af-f7b93adc1b06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:54.703482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-bd606434-8988-4547-8c28-f06be2f28ed7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:26:54.703482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:54.703482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:54.703482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:54.703482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:54.703482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:54.703482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:54.703482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:54.703482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:54.703482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:54.703482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:54.703482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:54.703482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:26:54.703482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:26:54.703482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:26:54.703482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:26:54.704429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:26:54.704429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:26:54.704429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:26:54.704429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:26:54.704429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:54.704429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:54.704429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:54.704429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:54.704429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:54.704429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:54.704429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:54.704429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:54.704429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:54.704429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:54.704429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:54.704429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:54.705208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:54.705208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:54.705208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:54.705208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:54.705208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:26:54.705208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:26:54.705208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:26:54.705208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:26:54.705208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:26:54.705208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:26:54.705208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:26:54.705208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:26:54.705813 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c08830-feb7-4ae0-81af-f7b93adc1b06 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:54.705910 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 219/446] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:26:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:26:54.710364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd606434-8988-4547-8c28-f06be2f28ed7 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:54.716847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:54.827915 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 228/447] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:26:54.833748 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c08830-feb7-4ae0-81af-f7b93adc1b06 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:54.835963 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd606434-8988-4547-8c28-f06be2f28ed7 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:54.869652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736acdd7-687f-4696-814b-d81e29672e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:54.870307 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736acdd7-687f-4696-814b-d81e29672e39 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:54.870619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736acdd7-687f-4696-814b-d81e29672e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:54.870892 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736acdd7-687f-4696-814b-d81e29672e39 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:54.871661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736acdd7-687f-4696-814b-d81e29672e39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:54.871960 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736acdd7-687f-4696-814b-d81e29672e39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:54.873320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736acdd7-687f-4696-814b-d81e29672e39 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:54.881545 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 220/448] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:26:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:54.887651 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736acdd7-687f-4696-814b-d81e29672e39 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:54.890282 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:54.931786 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c08830-feb7-4ae0-81af-f7b93adc1b06 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:54.941381 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:54.966318 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eff0f2-1bb5-4da9-9ae6-398e94e65840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:54.967397 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eff0f2-1bb5-4da9-9ae6-398e94e65840 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:54.967928 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eff0f2-1bb5-4da9-9ae6-398e94e65840 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:54.968197 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eff0f2-1bb5-4da9-9ae6-398e94e65840 None admin] PATH_INFO: `/v3/projects/4d7f983bf3f84850a4a59357572371d9` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:54.969377 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eff0f2-1bb5-4da9-9ae6-398e94e65840 None admin] RBAC: Authorizing `identity:get_project(project_id=4d7f983bf3f84850a4a59357572371d9)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:54.970838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eff0f2-1bb5-4da9-9ae6-398e94e65840 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:54.972513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 229/449] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:54 2022] GET /identity/v3/projects/4d7f983bf3f84850a4a59357572371d9 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:54.977726 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eff0f2-1bb5-4da9-9ae6-398e94e65840 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:54.988878 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736acdd7-687f-4696-814b-d81e29672e39 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:55.021881 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7f4e1-2014-4a29-8a89-ae69a881efc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:55.022482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7f4e1-2014-4a29-8a89-ae69a881efc1 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:55.022768 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7f4e1-2014-4a29-8a89-ae69a881efc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:55.023039 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7f4e1-2014-4a29-8a89-ae69a881efc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:55.078257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eff0f2-1bb5-4da9-9ae6-398e94e65840 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:55.088620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de7f4e1-2014-4a29-8a89-ae69a881efc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:55.091012 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de7f4e1-2014-4a29-8a89-ae69a881efc1 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:55.093271 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:55.112719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108925c3-0f66-4d9b-888c-872e77ddba98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:55.113711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108925c3-0f66-4d9b-888c-872e77ddba98 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:55.113993 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108925c3-0f66-4d9b-888c-872e77ddba98 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:55.114267 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108925c3-0f66-4d9b-888c-872e77ddba98 None admin] PATH_INFO: `/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:55.115453 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108925c3-0f66-4d9b-888c-872e77ddba98 None admin] RBAC: Authorizing `identity:get_group(group_id=9f7e220e2ba14938ae0cac70c096cb8b)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:55.116894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108925c3-0f66-4d9b-888c-872e77ddba98 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:55.118520 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 230/450] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Jun 27 18:26:54 2022] GET /identity/v3/groups/9f7e220e2ba14938ae0cac70c096cb8b => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:55.124377 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108925c3-0f66-4d9b-888c-872e77ddba98 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:55.169569 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:55.192766 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 221/451] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:26:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 18:26:55.198309 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7f4e1-2014-4a29-8a89-ae69a881efc1 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:55.223735 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108925c3-0f66-4d9b-888c-872e77ddba98 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:55.257604 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a856a-aa47-45b7-9b69-4b739467e020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:55.258281 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a856a-aa47-45b7-9b69-4b739467e020 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:55.258594 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a856a-aa47-45b7-9b69-4b739467e020 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:55.258875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a856a-aa47-45b7-9b69-4b739467e020 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:55.259705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a856a-aa47-45b7-9b69-4b739467e020 None admin] RBAC: Adding query filter params (group.id=9f7e220e2ba14938ae0cac70c096cb8b, role.id=44d7d6bfe7eb441bbc6a301a5b6784ad, scope.project.id=4d7f983bf3f84850a4a59357572371d9) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:55.260017 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a856a-aa47-45b7-9b69-4b739467e020 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:55.261455 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a856a-aa47-45b7-9b69-4b739467e020 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:55.267316 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 231/452] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Jun 27 18:26:55 2022] GET /identity/v3/role_assignments?group.id=9f7e220e2ba14938ae0cac70c096cb8b&scope.project.id=4d7f983bf3f84850a4a59357572371d9&role.id=44d7d6bfe7eb441bbc6a301a5b6784ad => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:55.300805 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7f4e1-2014-4a29-8a89-ae69a881efc1 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:55.302673 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:55.337910 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682bafe-4083-418d-82c4-136c77abf87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:55.338814 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682bafe-4083-418d-82c4-136c77abf87c None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:55.339099 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682bafe-4083-418d-82c4-136c77abf87c None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:55.339373 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b682bafe-4083-418d-82c4-136c77abf87c None admin] PATH_INFO: `/v3/projects/d9ffe361a6094e199bb35c633199d02f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:55.341165 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682bafe-4083-418d-82c4-136c77abf87c None admin] RBAC: Authorizing `identity:get_project(project_id=d9ffe361a6094e199bb35c633199d02f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:55.342695 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b682bafe-4083-418d-82c4-136c77abf87c None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:55.345334 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 222/453] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:26:55 2022] GET /identity/v3/projects/d9ffe361a6094e199bb35c633199d02f => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:55.350495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a856a-aa47-45b7-9b69-4b739467e020 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:55.455699 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a856a-aa47-45b7-9b69-4b739467e020 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:55.465611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:55.491703 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ee140-7669-4124-9dc3-4c353aa852cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:55.493225 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ee140-7669-4124-9dc3-4c353aa852cb None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:55.493225 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ee140-7669-4124-9dc3-4c353aa852cb None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:55.493407 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ee140-7669-4124-9dc3-4c353aa852cb None admin] PATH_INFO: `/v3/groups/6e27a65d60a144a88caed67204d4a9bb` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:55.494482 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ee140-7669-4124-9dc3-4c353aa852cb None admin] RBAC: Authorizing `identity:get_group(group_id=6e27a65d60a144a88caed67204d4a9bb)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:55.496145 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ee140-7669-4124-9dc3-4c353aa852cb None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:55.498060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 232/454] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Jun 27 18:26:55 2022] GET /identity/v3/groups/6e27a65d60a144a88caed67204d4a9bb => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:55.503942 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682bafe-4083-418d-82c4-136c77abf87c None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:55.608285 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b682bafe-4083-418d-82c4-136c77abf87c None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:55.641441 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d6a2f-8126-4829-910e-646f364072b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:55.642079 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d6a2f-8126-4829-910e-646f364072b6 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:55.642356 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d6a2f-8126-4829-910e-646f364072b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:55.642724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d6a2f-8126-4829-910e-646f364072b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:55.643544 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d6a2f-8126-4829-910e-646f364072b6 None admin] RBAC: Adding query filter params (group.id=6e27a65d60a144a88caed67204d4a9bb, role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f, scope.project.id=d9ffe361a6094e199bb35c633199d02f) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:55.643853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d6a2f-8126-4829-910e-646f364072b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:55.645266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d6a2f-8126-4829-910e-646f364072b6 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:55.650906 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 223/455] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Jun 27 18:26:55 2022] GET /identity/v3/role_assignments?group.id=6e27a65d60a144a88caed67204d4a9bb&scope.project.id=d9ffe361a6094e199bb35c633199d02f&role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:58.277819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdac53-9a37-4081-a1f0-a29495d98438 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:58.278715 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdac53-9a37-4081-a1f0-a29495d98438 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:58.279135 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdac53-9a37-4081-a1f0-a29495d98438 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:58.280905 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 233/456] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:58.300570 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6aab80-be18-45aa-84cb-30db811fc806 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:58.301108 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6aab80-be18-45aa-84cb-30db811fc806 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:58.301108 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6aab80-be18-45aa-84cb-30db811fc806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:58.323471 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-5f6aab80-be18-45aa-84cb-30db811fc806 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:58.325600 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-5f6aab80-be18-45aa-84cb-30db811fc806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:58.362862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:58.414966 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 224/457] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:58.462883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cdac53-9a37-4081-a1f0-a29495d98438 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:58.534499 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bcc71-397d-4ed1-a72d-e08b1d0763a2 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:58.535007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bcc71-397d-4ed1-a72d-e08b1d0763a2 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:58.535443 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bcc71-397d-4ed1-a72d-e08b1d0763a2 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:58.537342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 225/458] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:58.543792 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0500c8-a1a2-483f-9d81-807717ed8aca None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:58.544188 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0500c8-a1a2-483f-9d81-807717ed8aca None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:58.544467 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0500c8-a1a2-483f-9d81-807717ed8aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:58.576064 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-dd0500c8-a1a2-483f-9d81-807717ed8aca None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:58.578075 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-dd0500c8-a1a2-483f-9d81-807717ed8aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:58.597412 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cdac53-9a37-4081-a1f0-a29495d98438 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:58.635324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ed736-6030-47bf-9e1e-4289458e3d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:58.636355 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ed736-6030-47bf-9e1e-4289458e3d2d None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:58.636875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ed736-6030-47bf-9e1e-4289458e3d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:58.637387 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ed736-6030-47bf-9e1e-4289458e3d2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:58.648343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ed736-6030-47bf-9e1e-4289458e3d2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:58.650065 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ed736-6030-47bf-9e1e-4289458e3d2d None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:58.667608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 226/459] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:26:58.669524 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30dc8f-efbd-4153-8fd1-f5920124186c None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:58.669812 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30dc8f-efbd-4153-8fd1-f5920124186c None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:58.670086 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30dc8f-efbd-4153-8fd1-f5920124186c None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:58.670514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-726ed736-6030-47bf-9e1e-4289458e3d2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:26:58.670514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:58.670514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:58.670514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:58.670514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:58.670514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:58.670514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:58.670514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:58.670514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:58.670514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:58.670514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:58.670514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:58.670514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:26:58.670514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:26:58.670514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:26:58.671606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:26:58.671606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:58.671606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:58.671606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:58.671606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:58.671606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:58.671606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:58.671606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:58.671606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:58.671606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:58.671606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:58.671606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:58.671606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:58.671606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:58.671606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:58.671606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:58.671606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:58.672547 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:26:58.672547 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:26:58.672547 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:26:58.672547 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:58.674413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 234/460] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:26:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:58.679541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ed736-6030-47bf-9e1e-4289458e3d2d None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:58.679754 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 227/461] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:58 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:58.685393 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d41dbf-78f0-47f9-937e-5cc9209ee203 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:58.685862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d41dbf-78f0-47f9-937e-5cc9209ee203 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:58.686290 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d41dbf-78f0-47f9-937e-5cc9209ee203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:58.704274 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-a1d41dbf-78f0-47f9-937e-5cc9209ee203 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:58.705677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-a1d41dbf-78f0-47f9-937e-5cc9209ee203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:58.738122 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:58.788747 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 228/462] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:26:58.789507 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d41dbf-78f0-47f9-937e-5cc9209ee203 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:58.811749 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ed736-6030-47bf-9e1e-4289458e3d2d None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:58.824559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:58.846986 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85096b9e-67b9-49f2-b9d4-2860632e4001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:58.847565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85096b9e-67b9-49f2-b9d4-2860632e4001 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:58.847847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85096b9e-67b9-49f2-b9d4-2860632e4001 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:58.848121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85096b9e-67b9-49f2-b9d4-2860632e4001 None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:58.848849 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85096b9e-67b9-49f2-b9d4-2860632e4001 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:58.849157 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85096b9e-67b9-49f2-b9d4-2860632e4001 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:58.850546 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85096b9e-67b9-49f2-b9d4-2860632e4001 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:58.868924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 235/463] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:26:58.870049 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2ce4f-35bb-4afa-ba6e-f348d57d4184 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:58.870333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2ce4f-35bb-4afa-ba6e-f348d57d4184 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:58.870638 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2ce4f-35bb-4afa-ba6e-f348d57d4184 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:58.872048 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 236/464] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:58.872612 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2ce4f-35bb-4afa-ba6e-f348d57d4184 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:58.909691 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d41dbf-78f0-47f9-937e-5cc9209ee203 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:58.946039 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b0db2-1245-4969-ab8b-88d2b45e187e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:58.946853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b0db2-1245-4969-ab8b-88d2b45e187e None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:58.947134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b0db2-1245-4969-ab8b-88d2b45e187e None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:58.947406 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b0db2-1245-4969-ab8b-88d2b45e187e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:58.958470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b0db2-1245-4969-ab8b-88d2b45e187e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:58.959931 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b0db2-1245-4969-ab8b-88d2b45e187e None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:58.969574 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:58.978202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-0a4b0db2-1245-4969-ab8b-88d2b45e187e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:26:58.978202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:58.978202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:58.978202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:58.978202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:58.978202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:58.978202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:58.978202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:58.978202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:58.978202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:58.978202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:58.978202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:58.978202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:26:58.978202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:26:58.978202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:26:58.979229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:26:58.979229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:58.979229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:58.979229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:58.979229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:58.979229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:58.979229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:58.979229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:58.979229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:58.979229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:58.979229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:58.979229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:58.979229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:58.979229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:58.979229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:58.979229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:58.979229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:58.980475 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:26:58.980475 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:26:58.980475 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:26:58.980475 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:26:58.980475 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 229/465] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:26:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:58.980475 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b0db2-1245-4969-ab8b-88d2b45e187e None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:58.988141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2ce4f-35bb-4afa-ba6e-f348d57d4184 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:59.025873 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c1361-4aa5-49b3-ab0b-a34493325c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:59.027000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c1361-4aa5-49b3-ab0b-a34493325c23 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:59.028166 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c1361-4aa5-49b3-ab0b-a34493325c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:59.028454 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c1361-4aa5-49b3-ab0b-a34493325c23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:59.039784 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c1361-4aa5-49b3-ab0b-a34493325c23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:59.041190 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c1361-4aa5-49b3-ab0b-a34493325c23 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:59.060595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-c22c1361-4aa5-49b3-ab0b-a34493325c23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:26:59.060595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:59.060595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:59.060595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:59.060595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:59.060595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:59.060595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:59.060595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:59.060595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:59.060595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:59.060595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:59.060595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:59.060595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:26:59.060595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:26:59.060595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:26:59.061675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:26:59.061675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:59.061675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:59.061675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:59.061675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:59.061675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:59.061675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:59.061675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:59.061675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:59.061675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:59.061675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:59.061675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:59.061675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:59.061675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:59.061675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:59.061675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:59.061675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:59.062758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:26:59.062758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:26:59.062758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:26:59.062758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:59.066892 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:59.067491 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 237/466] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:26:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:59.068861 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c26734-9219-4bb5-8a09-ba91a17c4c82 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:59.069147 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c26734-9219-4bb5-8a09-ba91a17c4c82 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:59.069422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c26734-9219-4bb5-8a09-ba91a17c4c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:59.088998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-78c26734-9219-4bb5-8a09-ba91a17c4c82 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:59.090192 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-78c26734-9219-4bb5-8a09-ba91a17c4c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:59.090659 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b0db2-1245-4969-ab8b-88d2b45e187e None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:59.128853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b52da9-8749-4849-9591-558431420ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:59.129525 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b52da9-8749-4849-9591-558431420ce4 None admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:59.129806 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b52da9-8749-4849-9591-558431420ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:59.130083 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b52da9-8749-4849-9591-558431420ce4 None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:59.131450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b52da9-8749-4849-9591-558431420ce4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:59.133337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b52da9-8749-4849-9591-558431420ce4 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:59.161342 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 238/467] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:59.162255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c26734-9219-4bb5-8a09-ba91a17c4c82 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:59.223030 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 230/468] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Jun 27 18:26:58 2022] POST /identity/v3/users => generated 246 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:26:59.223848 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b52da9-8749-4849-9591-558431420ce4 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:59.247028 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:59.283030 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c26734-9219-4bb5-8a09-ba91a17c4c82 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:59.324386 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c5ea5-9f2d-4f30-adee-64dd869df412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:59.324996 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c5ea5-9f2d-4f30-adee-64dd869df412 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:59.325280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c5ea5-9f2d-4f30-adee-64dd869df412 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:59.325559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c5ea5-9f2d-4f30-adee-64dd869df412 None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:59.326323 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c5ea5-9f2d-4f30-adee-64dd869df412 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:59.326663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c5ea5-9f2d-4f30-adee-64dd869df412 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:59.328541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c5ea5-9f2d-4f30-adee-64dd869df412 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:59.346124 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 239/469] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:59.346124 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c5ea5-9f2d-4f30-adee-64dd869df412 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:59.350575 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b52da9-8749-4849-9591-558431420ce4 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:59.385006 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:59.391788 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fe3c20-3e9e-4eef-a1d1-6aeba6e00a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:59.394741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe3c20-3e9e-4eef-a1d1-6aeba6e00a23 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:59.394741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe3c20-3e9e-4eef-a1d1-6aeba6e00a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:59.394741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe3c20-3e9e-4eef-a1d1-6aeba6e00a23 None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:59.394741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fe3c20-3e9e-4eef-a1d1-6aeba6e00a23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:59.394741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fe3c20-3e9e-4eef-a1d1-6aeba6e00a23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:59.395414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fe3c20-3e9e-4eef-a1d1-6aeba6e00a23 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:59.409776 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 231/470] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:59.410799 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fe3c20-3e9e-4eef-a1d1-6aeba6e00a23 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:59.465351 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c5ea5-9f2d-4f30-adee-64dd869df412 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:59.498588 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:59.504527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be58a0eb-0dd2-4101-ad31-deecd779793f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:59.505315 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58a0eb-0dd2-4101-ad31-deecd779793f None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:59.505624 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58a0eb-0dd2-4101-ad31-deecd779793f None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:59.505857 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58a0eb-0dd2-4101-ad31-deecd779793f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:59.517096 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be58a0eb-0dd2-4101-ad31-deecd779793f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:59.518678 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be58a0eb-0dd2-4101-ad31-deecd779793f None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:59.522602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fe3c20-3e9e-4eef-a1d1-6aeba6e00a23 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:59.536026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-be58a0eb-0dd2-4101-ad31-deecd779793f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:26:59.536026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:26:59.536026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:26:59.536026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:26:59.536026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:26:59.536026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:26:59.536026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:26:59.536026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:26:59.536026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:26:59.536026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:26:59.536026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:26:59.536026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:26:59.536026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:26:59.536026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:26:59.536026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:26:59.536896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:26:59.536896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:26:59.536896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:26:59.536896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:26:59.536896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:26:59.536896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:26:59.536896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:26:59.536896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:26:59.536896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:26:59.536896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:26:59.536896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:26:59.536896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:26:59.536896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:26:59.536896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:26:59.536896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:26:59.536896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:26:59.536896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:26:59.537720 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:26:59.537720 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:26:59.537720 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:26:59.537720 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:26:59.537720 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 240/471] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:26:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:26:59.538647 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfe70c-1d59-415e-8d35-bea74af153de None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:59.538753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfe70c-1d59-415e-8d35-bea74af153de None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:59.539036 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfe70c-1d59-415e-8d35-bea74af153de None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:59.540473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 241/472] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:26:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:26:59.544260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfe70c-1d59-415e-8d35-bea74af153de None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:59.564968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2788951-4683-43fa-836a-0e03d1f7dc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:59.565932 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2788951-4683-43fa-836a-0e03d1f7dc9b None admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:59.566223 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2788951-4683-43fa-836a-0e03d1f7dc9b None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:59.566538 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2788951-4683-43fa-836a-0e03d1f7dc9b None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:59.567239 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2788951-4683-43fa-836a-0e03d1f7dc9b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:59.569131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2788951-4683-43fa-836a-0e03d1f7dc9b None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:59.651309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfe70c-1d59-415e-8d35-bea74af153de None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:59.680204 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 232/473] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Jun 27 18:26:59 2022] POST /identity/v3/users => generated 248 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:26:59.681352 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2788951-4683-43fa-836a-0e03d1f7dc9b None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:59.695473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97247319-d452-4a8a-8653-b50f2cc6c6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:59.695875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97247319-d452-4a8a-8653-b50f2cc6c6f3 None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:59.696205 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97247319-d452-4a8a-8653-b50f2cc6c6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:59.696485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97247319-d452-4a8a-8653-b50f2cc6c6f3 None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:59.697231 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97247319-d452-4a8a-8653-b50f2cc6c6f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:59.698787 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97247319-d452-4a8a-8653-b50f2cc6c6f3 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:59.750932 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:59.789304 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 242/474] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Jun 27 18:26:59 2022] POST /identity/v3/users => generated 248 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:26:59.789304 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5c201-7ea6-47f8-9967-5e907e258af4 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:59.789304 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5c201-7ea6-47f8-9967-5e907e258af4 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:59.789304 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5c201-7ea6-47f8-9967-5e907e258af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:59.821430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-97b5c201-7ea6-47f8-9967-5e907e258af4 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:26:59.822732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-97b5c201-7ea6-47f8-9967-5e907e258af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:26:59.848486 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2788951-4683-43fa-836a-0e03d1f7dc9b None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:26:59.875896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:26:59.895248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37c434-61dd-43ca-beed-ee34ec9adad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:26:59.895923 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37c434-61dd-43ca-beed-ee34ec9adad9 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:26:59.898922 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37c434-61dd-43ca-beed-ee34ec9adad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:26:59.898922 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37c434-61dd-43ca-beed-ee34ec9adad9 None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:26:59.898922 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37c434-61dd-43ca-beed-ee34ec9adad9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:26:59.898922 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37c434-61dd-43ca-beed-ee34ec9adad9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:26:59.899228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37c434-61dd-43ca-beed-ee34ec9adad9 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:26:59.914158 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 233/475] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:26:59.919536 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37c434-61dd-43ca-beed-ee34ec9adad9 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:26:59.931955 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 243/476] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:26:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:26:59.962078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5c201-7ea6-47f8-9967-5e907e258af4 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:00.058718 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37c434-61dd-43ca-beed-ee34ec9adad9 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:00.070272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:00.098768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5c201-7ea6-47f8-9967-5e907e258af4 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:00.102888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3533286-7a12-44f2-9602-78543e328837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:00.103559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3533286-7a12-44f2-9602-78543e328837 None admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:00.103843 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3533286-7a12-44f2-9602-78543e328837 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:00.104123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3533286-7a12-44f2-9602-78543e328837 None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:00.104844 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3533286-7a12-44f2-9602-78543e328837 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:00.107139 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3533286-7a12-44f2-9602-78543e328837 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:00.143515 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144b3da-c4f9-41fd-a829-ddf200765337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:00.144644 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144b3da-c4f9-41fd-a829-ddf200765337 None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:00.145175 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144b3da-c4f9-41fd-a829-ddf200765337 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:00.145644 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144b3da-c4f9-41fd-a829-ddf200765337 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:00.146656 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144b3da-c4f9-41fd-a829-ddf200765337 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:00.150223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144b3da-c4f9-41fd-a829-ddf200765337 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:00.167699 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 234/477] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Jun 27 18:26:59 2022] POST /identity/v3/users => generated 249 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:27:00.172117 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.sql.core [None req-f144b3da-c4f9-41fd-a829-ddf200765337 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 27 18:27:00.172117 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 27 18:27:00.172117 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [parameters: {'id': '87a238ad8b9747f188a77717fc4797b9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 27 18:27:00.172117 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80570) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 27 18:27:00.190995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-f144b3da-c4f9-41fd-a829-ddf200765337 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 27 18:27:00.190995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:00.190995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 18:27:00.190995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 18:27:00.190995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 18:27:00.190995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 18:27:00.190995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 18:27:00.190995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 18:27:00.190995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 18:27:00.190995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application conn.query(q) Jun 27 18:27:00.190995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 18:27:00.190995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 18:27:00.190995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 18:27:00.190995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application result.read() Jun 27 18:27:00.190995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 18:27:00.192137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 18:27:00.192137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 18:27:00.192137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 18:27:00.192137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 18:27:00.192137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 18:27:00.192137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 18:27:00.192137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 18:27:00.192137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 27 18:27:00.192137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:00.192137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 27 18:27:00.192137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:00.192137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:00.192137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 27 18:27:00.192137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 27 18:27:00.192137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 27 18:27:00.192137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return ref.to_dict() Jun 27 18:27:00.192137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 18:27:00.192137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application next(self.gen) Jun 27 18:27:00.193257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 27 18:27:00.193257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application yield resource Jun 27 18:27:00.193257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 18:27:00.193257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application next(self.gen) Jun 27 18:27:00.193257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 27 18:27:00.193257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self.session.rollback() Jun 27 18:27:00.193257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 27 18:27:00.193257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self.force_reraise() Jun 27 18:27:00.193257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 27 18:27:00.193257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise self.value Jun 27 18:27:00.193257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 27 18:27:00.193257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 27 18:27:00.193257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 27 18:27:00.193257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application session.commit() Jun 27 18:27:00.193257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 27 18:27:00.193257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 27 18:27:00.193257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 27 18:27:00.194458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._prepare_impl() Jun 27 18:27:00.194458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 27 18:27:00.194458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self.session.flush() Jun 27 18:27:00.194458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 27 18:27:00.194458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._flush(objects) Jun 27 18:27:00.194458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 27 18:27:00.194458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 27 18:27:00.194458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 27 18:27:00.194458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application compat.raise_( Jun 27 18:27:00.194458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 18:27:00.194458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception Jun 27 18:27:00.194458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 27 18:27:00.194458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application flush_context.execute() Jun 27 18:27:00.194458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 27 18:27:00.194458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rec.execute(self) Jun 27 18:27:00.194458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 27 18:27:00.194458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 27 18:27:00.195632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 27 18:27:00.195632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 27 18:27:00.195632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 27 18:27:00.195632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 27 18:27:00.195632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 27 18:27:00.195632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 27 18:27:00.195632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 27 18:27:00.195632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 27 18:27:00.195632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 27 18:27:00.195632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 27 18:27:00.195632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 27 18:27:00.195632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 27 18:27:00.195632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 27 18:27:00.195632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 27 18:27:00.195632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 18:27:00.195632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception Jun 27 18:27:00.196659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 18:27:00.196659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 18:27:00.196659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 18:27:00.196659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 18:27:00.196659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 18:27:00.196659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 18:27:00.196659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 18:27:00.196659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application conn.query(q) Jun 27 18:27:00.196659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 18:27:00.196659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 18:27:00.196659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 18:27:00.196659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application result.read() Jun 27 18:27:00.196659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 18:27:00.196659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 18:27:00.196659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 18:27:00.196659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 18:27:00.197470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 18:27:00.197470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 18:27:00.197470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 18:27:00.197470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 18:27:00.197470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 27 18:27:00.197470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 27 18:27:00.197470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [parameters: {'id': '87a238ad8b9747f188a77717fc4797b9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 27 18:27:00.197470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 27 18:27:00.197470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:00.197470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:00.197470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:00.197470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:00.197470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:00.197470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:00.197470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:00.197470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:00.198302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:00.198302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:00.198302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:00.198302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:00.198302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:00.198302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:00.198302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 27 18:27:00.198302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 27 18:27:00.198302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:00.198302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:00.198302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 27 18:27:00.198302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 27 18:27:00.198302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 27 18:27:00.198302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 27 18:27:00.198302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 27 18:27:00.198302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:00.202249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 244/478] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Jun 27 18:26:59 2022] POST /identity/v3/roles => generated 144 bytes in 241 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:27:00.211731 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3533286-7a12-44f2-9602-78543e328837 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:00.336264 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3533286-7a12-44f2-9602-78543e328837 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:00.378455 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac2978-3525-4964-a529-5b2ace41ea69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:00.379968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac2978-3525-4964-a529-5b2ace41ea69 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:00.380395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac2978-3525-4964-a529-5b2ace41ea69 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:00.380814 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac2978-3525-4964-a529-5b2ace41ea69 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:00.392295 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac2978-3525-4964-a529-5b2ace41ea69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:00.400741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-caac2978-3525-4964-a529-5b2ace41ea69 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:27:00.400741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:00.400741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:00.400741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:00.400741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:00.400741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:00.400741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:00.400741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:00.400741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:00.400741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:00.400741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:00.400741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:00.400741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:00.400741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:00.400741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:00.400741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:00.401801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:00.401801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:00.401801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:00.401801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:00.401801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:00.401801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:00.401801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:00.401801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:00.401801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:00.401801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:00.401801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:00.401801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:00.401801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:00.401801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:00.401801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:00.401801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:00.402862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:00.402862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:00.402862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:00.402862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:00.402862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:00.402862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:00.402862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:00.402862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:00.402862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:00.402862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:00.402862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 18:27:00.402862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:00.405173 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 235/479] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Jun 27 18:27:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:00.410355 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144b3da-c4f9-41fd-a829-ddf200765337 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:00.450570 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:00.526812 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144b3da-c4f9-41fd-a829-ddf200765337 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:00.566518 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd918901-646f-41d1-8fb5-a744a2a0a35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:00.567424 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd918901-646f-41d1-8fb5-a744a2a0a35f None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:00.567874 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd918901-646f-41d1-8fb5-a744a2a0a35f None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:00.568290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd918901-646f-41d1-8fb5-a744a2a0a35f None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:00.569203 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd918901-646f-41d1-8fb5-a744a2a0a35f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:00.569689 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd918901-646f-41d1-8fb5-a744a2a0a35f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:00.571263 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd918901-646f-41d1-8fb5-a744a2a0a35f None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:00.580368 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 245/480] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:01.566979 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd945353-3c85-4f17-8988-1cf7000904f2 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:01.567867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd945353-3c85-4f17-8988-1cf7000904f2 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:01.568394 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd945353-3c85-4f17-8988-1cf7000904f2 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:01.570237 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 236/481] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:01.575398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645ba4e-a035-4274-8552-0cd956c17c13 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:01.576200 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645ba4e-a035-4274-8552-0cd956c17c13 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:01.576726 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645ba4e-a035-4274-8552-0cd956c17c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:01.603313 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-c645ba4e-a035-4274-8552-0cd956c17c13 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:01.604961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c645ba4e-a035-4274-8552-0cd956c17c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:01.695733 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 246/482] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:27:01.731877 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd945353-3c85-4f17-8988-1cf7000904f2 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:01.771563 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:01.876047 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd945353-3c85-4f17-8988-1cf7000904f2 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:01.921030 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148dbd4-dc15-4bcb-b080-5b271021bad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:01.921827 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148dbd4-dc15-4bcb-b080-5b271021bad6 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:01.922106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148dbd4-dc15-4bcb-b080-5b271021bad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:01.922376 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148dbd4-dc15-4bcb-b080-5b271021bad6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:01.938686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c148dbd4-dc15-4bcb-b080-5b271021bad6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:01.946435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-c148dbd4-dc15-4bcb-b080-5b271021bad6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:01.946435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:01.946435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:01.946435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:01.946435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:01.946435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:01.946435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:01.946435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:01.946435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:01.946435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:01.946435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:01.946435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:01.946435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:01.946435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:01.946435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:01.946435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:01.947328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:01.947328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:01.947328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:01.947328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:01.947328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:01.947328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:01.947328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:01.947328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:01.947328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:01.947328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:01.947328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:01.947328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:01.947328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:01.947328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:01.947328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:01.947328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:01.948183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:01.948183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:01.948183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:01.948183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:01.948183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:01.948183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:01.948183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:01.948183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:01.948183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:01.948183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:01.948183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:01.948183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:01.952211 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 237/483] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:27:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:01.954026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645ba4e-a035-4274-8552-0cd956c17c13 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:02.001947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:02.123437 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645ba4e-a035-4274-8552-0cd956c17c13 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:02.181236 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad99851-c7c9-4cce-a454-06964cbaa46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:02.181831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad99851-c7c9-4cce-a454-06964cbaa46c None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:02.182113 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad99851-c7c9-4cce-a454-06964cbaa46c None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:02.182408 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad99851-c7c9-4cce-a454-06964cbaa46c None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:02.183708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad99851-c7c9-4cce-a454-06964cbaa46c None admin] RBAC: Adding query filter params (name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:02.184139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad99851-c7c9-4cce-a454-06964cbaa46c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:02.185527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad99851-c7c9-4cce-a454-06964cbaa46c None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:02.197949 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 247/484] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Jun 27 18:27:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:02.204149 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148dbd4-dc15-4bcb-b080-5b271021bad6 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:02.340792 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:02.357886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c148dbd4-dc15-4bcb-b080-5b271021bad6 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:02.363360 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34607a6-a796-4a76-b7e4-eff2f1c45e92 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:02.363667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34607a6-a796-4a76-b7e4-eff2f1c45e92 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:02.363937 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34607a6-a796-4a76-b7e4-eff2f1c45e92 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:02.365661 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 248/485] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:02.370459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a40d40-b63c-4335-861e-5e2725e44c5a None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:02.370793 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a40d40-b63c-4335-861e-5e2725e44c5a None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:02.371119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a40d40-b63c-4335-861e-5e2725e44c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:02.397793 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73180ca2-3f31-4808-9977-1fe8803d7f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:02.398897 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73180ca2-3f31-4808-9977-1fe8803d7f4a None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:02.399350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73180ca2-3f31-4808-9977-1fe8803d7f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:02.399792 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73180ca2-3f31-4808-9977-1fe8803d7f4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:02.411394 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-f7a40d40-b63c-4335-861e-5e2725e44c5a None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:02.411774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73180ca2-3f31-4808-9977-1fe8803d7f4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:02.412611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-f7a40d40-b63c-4335-861e-5e2725e44c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:02.413580 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73180ca2-3f31-4808-9977-1fe8803d7f4a None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:02.436921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-73180ca2-3f31-4808-9977-1fe8803d7f4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:02.436921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:02.436921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:02.436921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:02.436921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:02.436921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:02.436921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:02.436921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:02.436921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:02.436921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:02.436921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:02.436921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:02.436921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:02.436921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:02.436921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:02.437809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:02.437809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:02.437809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:02.437809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:02.437809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:02.437809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:02.437809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:02.437809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:02.437809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:02.437809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:02.437809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:02.437809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:02.437809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:02.437809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:02.437809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:02.437809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:02.437809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:02.438671 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:02.438671 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:02.438671 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:02.438671 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:02.440639 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 238/486] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:02.447297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73180ca2-3f31-4808-9977-1fe8803d7f4a None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:02.550591 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 249/487] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:27:02.552072 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bf7d1-be1d-476c-aa9b-6c45135b1829 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:02.552352 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bf7d1-be1d-476c-aa9b-6c45135b1829 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:02.552639 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bf7d1-be1d-476c-aa9b-6c45135b1829 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:02.556099 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73180ca2-3f31-4808-9977-1fe8803d7f4a None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:02.557681 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 250/488] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:02.570499 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46168bc8-7142-4bed-9b5a-c048b76cc696 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:02.570791 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46168bc8-7142-4bed-9b5a-c048b76cc696 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:02.571075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46168bc8-7142-4bed-9b5a-c048b76cc696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:02.595870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-46168bc8-7142-4bed-9b5a-c048b76cc696 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:02.597753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-46168bc8-7142-4bed-9b5a-c048b76cc696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:02.609954 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526f410-047b-4908-ab44-04a6aa94f0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:02.610692 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526f410-047b-4908-ab44-04a6aa94f0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:02.610975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526f410-047b-4908-ab44-04a6aa94f0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:02.611248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526f410-047b-4908-ab44-04a6aa94f0e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:02.611784 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:02.611971 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1526f410-047b-4908-ab44-04a6aa94f0e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:02.612278 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1526f410-047b-4908-ab44-04a6aa94f0e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:02.614417 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1526f410-047b-4908-ab44-04a6aa94f0e9 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:02.617853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:02.631435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 239/489] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:02.632494 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526f410-047b-4908-ab44-04a6aa94f0e9 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:02.679524 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 251/490] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:27:02.680699 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46168bc8-7142-4bed-9b5a-c048b76cc696 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:02.796499 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526f410-047b-4908-ab44-04a6aa94f0e9 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:02.832779 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46168bc8-7142-4bed-9b5a-c048b76cc696 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:02.842330 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2d55c-87a8-4060-840c-e7b037c11a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:02.843159 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2d55c-87a8-4060-840c-e7b037c11a79 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:02.843441 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2d55c-87a8-4060-840c-e7b037c11a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:02.843717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2d55c-87a8-4060-840c-e7b037c11a79 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:02.858750 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2d55c-87a8-4060-840c-e7b037c11a79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:02.869422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-65b2d55c-87a8-4060-840c-e7b037c11a79 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:02.869422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:02.869422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:02.869422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:02.869422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:02.869422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:02.869422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:02.869422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:02.869422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:02.869422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:02.869422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:02.869422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:02.869422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:02.869422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:02.869422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:02.869422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:02.870317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:02.870317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:02.870317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:02.870317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:02.870317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:02.870317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:02.870317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:02.870317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:02.870317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:02.870317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:02.870317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:02.870317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:02.870317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:02.870317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:02.870317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:02.870317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:02.872269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:02.872269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:02.872269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:02.872269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:02.872269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:02.872269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:02.872269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:02.872269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:02.872269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:02.872269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:02.872269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:02.872269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:02.874446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 240/491] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:27:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:02.875462 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2d55c-87a8-4060-840c-e7b037c11a79 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:02.879317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce827b1f-fbb1-4379-9fa4-8f96dc6f8f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:02.880363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce827b1f-fbb1-4379-9fa4-8f96dc6f8f78 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:02.880837 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce827b1f-fbb1-4379-9fa4-8f96dc6f8f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:02.881283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce827b1f-fbb1-4379-9fa4-8f96dc6f8f78 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:02.890056 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce827b1f-fbb1-4379-9fa4-8f96dc6f8f78 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:02.891866 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce827b1f-fbb1-4379-9fa4-8f96dc6f8f78 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:02.908348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-ce827b1f-fbb1-4379-9fa4-8f96dc6f8f78 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 18:27:02.908348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:02.908348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:02.908348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:02.908348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:02.908348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:02.908348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:02.908348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:02.908348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:02.908348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:02.908348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:02.908348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:02.908348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:02.908348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:02.908348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:02.908348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:02.909279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:02.909279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:02.909279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:02.909279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:02.909279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:02.909279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:02.909279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:02.909279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:02.909279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:02.909279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:02.909279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:02.909279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:02.909279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:02.909279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:02.909279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:02.909279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:02.909279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:02.909279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:02.910158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:02.910158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:02.910158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:02.910158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:02.910158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:02.910158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:02.910158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:02.910158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:02.910158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 18:27:02.910158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:02.912761 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 252/492] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Jun 27 18:27:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:02.913751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce827b1f-fbb1-4379-9fa4-8f96dc6f8f78 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:02.923530 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:02.945907 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:03.006583 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2d55c-87a8-4060-840c-e7b037c11a79 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:03.025287 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce827b1f-fbb1-4379-9fa4-8f96dc6f8f78 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:03.045124 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2ce96-c633-403c-bdd0-91b73cc424aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:03.045908 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2ce96-c633-403c-bdd0-91b73cc424aa None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:03.046186 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2ce96-c633-403c-bdd0-91b73cc424aa None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:03.046483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2ce96-c633-403c-bdd0-91b73cc424aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:03.057272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb2ce96-c633-403c-bdd0-91b73cc424aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:03.062967 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8277689-7444-45c1-8e4e-212f8a915863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:03.063936 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8277689-7444-45c1-8e4e-212f8a915863 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:03.064356 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8277689-7444-45c1-8e4e-212f8a915863 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:03.064866 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8277689-7444-45c1-8e4e-212f8a915863 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:03.065723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-6cb2ce96-c633-403c-bdd0-91b73cc424aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:03.065723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:03.065723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:03.065723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:03.065723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:03.065723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:03.065723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:03.065723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:03.065723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:03.065723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:03.065723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:03.065723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:03.065723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:03.065723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:03.065723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:03.065723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:03.066786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:03.066786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:03.066786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:03.066786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:03.066786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:03.066786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:03.066786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:03.066786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:03.066786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:03.066786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:03.066786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:03.066786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:03.066786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:03.066786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:03.066786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:03.066786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:03.067615 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:03.067615 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:03.067615 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:03.067615 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:03.067615 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:03.067615 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:03.067615 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:03.067615 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:03.067615 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:03.067615 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:03.067615 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:03.067615 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:03.067615 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 241/493] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:27:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:03.068279 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2ce96-c633-403c-bdd0-91b73cc424aa None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:03.068424 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8277689-7444-45c1-8e4e-212f8a915863 None admin] RBAC: Adding query filter params (name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:03.068750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8277689-7444-45c1-8e4e-212f8a915863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:03.070132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8277689-7444-45c1-8e4e-212f8a915863 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:03.080384 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 253/494] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Jun 27 18:27:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:03.081671 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6962bf3-c47a-40a5-a1b2-925cbc3fd9cc None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:03.081957 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6962bf3-c47a-40a5-a1b2-925cbc3fd9cc None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:03.082260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6962bf3-c47a-40a5-a1b2-925cbc3fd9cc None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:03.083629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 254/495] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:03.084150 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6962bf3-c47a-40a5-a1b2-925cbc3fd9cc None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:03.156947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:03.192045 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2ce96-c633-403c-bdd0-91b73cc424aa None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:03.195321 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6962bf3-c47a-40a5-a1b2-925cbc3fd9cc None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:03.210168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:03.229774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517652c-6718-4cb0-9bb0-0290b7d7f6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:03.230921 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc89deb-74c9-4a64-8545-f37ab79877d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:03.231039 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517652c-6718-4cb0-9bb0-0290b7d7f6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:03.231255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517652c-6718-4cb0-9bb0-0290b7d7f6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:03.231529 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517652c-6718-4cb0-9bb0-0290b7d7f6d3 None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:03.231867 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc89deb-74c9-4a64-8545-f37ab79877d4 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:03.232102 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc89deb-74c9-4a64-8545-f37ab79877d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:03.232433 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc89deb-74c9-4a64-8545-f37ab79877d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:03.232618 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517652c-6718-4cb0-9bb0-0290b7d7f6d3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:03.232912 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517652c-6718-4cb0-9bb0-0290b7d7f6d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:03.233180 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc89deb-74c9-4a64-8545-f37ab79877d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:03.233486 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc89deb-74c9-4a64-8545-f37ab79877d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:03.234324 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517652c-6718-4cb0-9bb0-0290b7d7f6d3 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:03.234917 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc89deb-74c9-4a64-8545-f37ab79877d4 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:03.243404 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 255/496] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Jun 27 18:27:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:03.244532 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d52959-0649-405e-9625-4d910e3052c5 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:03.244817 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d52959-0649-405e-9625-4d910e3052c5 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:03.245092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d52959-0649-405e-9625-4d910e3052c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:03.262261 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-11d52959-0649-405e-9625-4d910e3052c5 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:03.263440 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-11d52959-0649-405e-9625-4d910e3052c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:03.273032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 242/497] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Jun 27 18:27:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:03.273860 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517652c-6718-4cb0-9bb0-0290b7d7f6d3 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:03.329965 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 256/498] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:27:03.331385 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934473ab-43d3-4ecf-be17-8e0fc75c06b5 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:03.331680 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934473ab-43d3-4ecf-be17-8e0fc75c06b5 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:03.331957 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934473ab-43d3-4ecf-be17-8e0fc75c06b5 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:03.333214 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 257/499] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:03.333895 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934473ab-43d3-4ecf-be17-8e0fc75c06b5 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:03.385868 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517652c-6718-4cb0-9bb0-0290b7d7f6d3 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:03.395868 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:03.420130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a0f86-d4c0-4a46-9d37-5a7793c115c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:03.420934 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a0f86-d4c0-4a46-9d37-5a7793c115c4 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:03.421218 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a0f86-d4c0-4a46-9d37-5a7793c115c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:03.421493 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a0f86-d4c0-4a46-9d37-5a7793c115c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:03.431513 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a0f86-d4c0-4a46-9d37-5a7793c115c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:03.432982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a0f86-d4c0-4a46-9d37-5a7793c115c4 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:03.438361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934473ab-43d3-4ecf-be17-8e0fc75c06b5 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:03.452316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-7c5a0f86-d4c0-4a46-9d37-5a7793c115c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:03.452316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:03.452316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:03.452316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:03.452316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:03.452316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:03.452316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:03.452316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:03.452316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:03.452316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:03.452316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:03.452316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:03.452316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:03.452316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:03.452316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:03.453247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:03.453247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:03.453247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:03.453247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:03.453247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:03.453247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:03.453247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:03.453247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:03.453247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:03.453247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:03.453247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:03.453247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:03.453247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:03.453247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:03.453247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:03.453247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:03.453247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:03.454076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:03.454076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:03.454076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:03.454076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:03.454076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 243/500] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:03.454345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a0f86-d4c0-4a46-9d37-5a7793c115c4 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:03.476397 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d6e49-b628-4731-b1ab-4447d94ac41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:03.477183 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d6e49-b628-4731-b1ab-4447d94ac41e None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:03.477465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d6e49-b628-4731-b1ab-4447d94ac41e None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:03.477742 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d6e49-b628-4731-b1ab-4447d94ac41e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:03.482267 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:03.488188 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d6e49-b628-4731-b1ab-4447d94ac41e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:03.489643 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d6e49-b628-4731-b1ab-4447d94ac41e None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:03.507991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-f24d6e49-b628-4731-b1ab-4447d94ac41e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:03.507991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:03.507991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:03.507991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:03.507991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:03.507991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:03.507991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:03.507991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:03.507991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:03.507991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:03.507991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:03.507991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:03.507991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:03.507991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:03.507991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:03.508856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:03.508856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:03.508856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:03.508856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:03.508856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:03.508856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:03.508856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:03.508856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:03.508856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:03.508856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:03.508856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:03.508856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:03.508856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:03.508856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:03.508856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:03.508856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:03.508856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:03.509689 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:03.509689 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:03.509689 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:03.509689 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:03.509689 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 258/501] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:03.510510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182926ee-d833-4d67-984a-a2b62bd78d9d None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:03.510656 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182926ee-d833-4d67-984a-a2b62bd78d9d None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:03.510932 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182926ee-d833-4d67-984a-a2b62bd78d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:03.527862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-182926ee-d833-4d67-984a-a2b62bd78d9d None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:03.529005 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-182926ee-d833-4d67-984a-a2b62bd78d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:03.555579 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a0f86-d4c0-4a46-9d37-5a7793c115c4 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:03.589250 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a714c-6cbb-4423-89f3-66288f8993f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:03.590108 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a714c-6cbb-4423-89f3-66288f8993f9 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:03.590416 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a714c-6cbb-4423-89f3-66288f8993f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:03.590830 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a714c-6cbb-4423-89f3-66288f8993f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:03.595760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 259/502] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:27:03.596584 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182926ee-d833-4d67-984a-a2b62bd78d9d None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:03.654665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:03.659884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a714c-6cbb-4423-89f3-66288f8993f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:03.661354 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a714c-6cbb-4423-89f3-66288f8993f9 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:03.688620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:03.710159 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182926ee-d833-4d67-984a-a2b62bd78d9d None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:03.745180 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4835b-0e1a-41f8-8e03-c2133902ab2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:03.747565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4835b-0e1a-41f8-8e03-c2133902ab2e None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:03.747565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4835b-0e1a-41f8-8e03-c2133902ab2e None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:03.747805 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4835b-0e1a-41f8-8e03-c2133902ab2e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:03.757867 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4835b-0e1a-41f8-8e03-c2133902ab2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:03.765552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-38d4835b-0e1a-41f8-8e03-c2133902ab2e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:03.765552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:03.765552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:03.765552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:03.765552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:03.765552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:03.765552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:03.765552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:03.765552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:03.765552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:03.765552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:03.765552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:03.765552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:03.765552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:03.765552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:03.765552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:03.766782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:03.766782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:03.766782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:03.766782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:03.766782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:03.766782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:03.766782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:03.766782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:03.766782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:03.766782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:03.766782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:03.766782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:03.766782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:03.766782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:03.766782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:03.766782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:03.767989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:03.767989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:03.767989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:03.767989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:03.767989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:03.767989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:03.767989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:03.767989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:03.767989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:03.767989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:03.767989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:03.767989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:03.768675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 244/503] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:27:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:27:03.768675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a714c-6cbb-4423-89f3-66288f8993f9 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:03.768829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 260/504] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:27:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:03.768829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4835b-0e1a-41f8-8e03-c2133902ab2e None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:03.867084 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a714c-6cbb-4423-89f3-66288f8993f9 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:03.868173 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4835b-0e1a-41f8-8e03-c2133902ab2e None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:03.869707 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:03.892209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:03.900176 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7ebd8-7333-4d9c-99ed-f68f4c8a6fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:03.900488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a66485-ce1a-4991-b386-6104dde287ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:03.900822 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7ebd8-7333-4d9c-99ed-f68f4c8a6fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:03.901104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7ebd8-7333-4d9c-99ed-f68f4c8a6fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:03.901203 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a66485-ce1a-4991-b386-6104dde287ac None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:03.901380 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7ebd8-7333-4d9c-99ed-f68f4c8a6fa3 None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:03.901496 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a66485-ce1a-4991-b386-6104dde287ac None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:03.901690 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a66485-ce1a-4991-b386-6104dde287ac None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:03.902736 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7ebd8-7333-4d9c-99ed-f68f4c8a6fa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:03.903021 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a66485-ce1a-4991-b386-6104dde287ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:03.903151 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7ebd8-7333-4d9c-99ed-f68f4c8a6fa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:03.903524 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a66485-ce1a-4991-b386-6104dde287ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:03.904442 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7ebd8-7333-4d9c-99ed-f68f4c8a6fa3 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:03.905172 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a66485-ce1a-4991-b386-6104dde287ac None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:03.915995 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 245/505] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:03.916776 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7ebd8-7333-4d9c-99ed-f68f4c8a6fa3 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:03.917628 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 261/506] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:03.918533 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a66485-ce1a-4991-b386-6104dde287ac None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:04.017922 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a66485-ce1a-4991-b386-6104dde287ac None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:04.018595 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7ebd8-7333-4d9c-99ed-f68f4c8a6fa3 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:04.051564 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5118cbf-a5d9-4644-b9f2-f126f7b93da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:04.052333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5118cbf-a5d9-4644-b9f2-f126f7b93da7 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:04.052438 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c09bc-7dbd-4ca3-8033-cd12fec8e7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:04.052613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5118cbf-a5d9-4644-b9f2-f126f7b93da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:04.052885 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5118cbf-a5d9-4644-b9f2-f126f7b93da7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:04.053205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c09bc-7dbd-4ca3-8033-cd12fec8e7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:04.053489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c09bc-7dbd-4ca3-8033-cd12fec8e7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:04.053771 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c09bc-7dbd-4ca3-8033-cd12fec8e7e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:04.062798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5118cbf-a5d9-4644-b9f2-f126f7b93da7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:04.063940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c09bc-7dbd-4ca3-8033-cd12fec8e7e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:04.064269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5118cbf-a5d9-4644-b9f2-f126f7b93da7 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:04.065391 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c09bc-7dbd-4ca3-8033-cd12fec8e7e0 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:04.082846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-d5118cbf-a5d9-4644-b9f2-f126f7b93da7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:04.082846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:04.082846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:04.082846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:04.082846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:04.082846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:04.082846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:04.082846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:04.082846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:04.082846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:04.082846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:04.082846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:04.082846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:04.082846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:04.082846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:04.083721 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:04.083721 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:04.083721 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:04.083721 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:04.083721 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:04.083721 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:04.083721 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:04.083721 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:04.083721 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:04.083721 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:04.083721 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:04.083721 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:04.083721 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:04.083721 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:04.083721 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:04.083721 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:04.083721 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:04.084887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:04.084887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:04.084887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:04.084887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:04.085215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-b12c09bc-7dbd-4ca3-8033-cd12fec8e7e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:04.085215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:04.085215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:04.085215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:04.085215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:04.085215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:04.085215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:04.085215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:04.085215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:04.085215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:04.085215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:04.085215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:04.085215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:04.085215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:04.085215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:04.085964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:04.085964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:04.085964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:04.085964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:04.085964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:04.085964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:04.085964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:04.085964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:04.085964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:04.085964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:04.085964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:04.085964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:04.085964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:04.085964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:04.085964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:04.085964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:04.085964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:04.087241 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:04.087241 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:04.087241 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:04.087241 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:04.087550 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 262/507] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:04.087631 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 246/508] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:04.087712 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5118cbf-a5d9-4644-b9f2-f126f7b93da7 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:04.087833 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c09bc-7dbd-4ca3-8033-cd12fec8e7e0 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:04.108494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:04.125538 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:04.184331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c09bc-7dbd-4ca3-8033-cd12fec8e7e0 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:04.192521 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5118cbf-a5d9-4644-b9f2-f126f7b93da7 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:04.217877 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0049f9-2e98-46d4-95e1-801724f1060b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:04.218927 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0049f9-2e98-46d4-95e1-801724f1060b None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:04.219202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0049f9-2e98-46d4-95e1-801724f1060b None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:04.219468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0049f9-2e98-46d4-95e1-801724f1060b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:04.225954 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfe101-25df-4768-a83b-25096712c1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:04.226277 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0049f9-2e98-46d4-95e1-801724f1060b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:04.226631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfe101-25df-4768-a83b-25096712c1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:04.226911 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfe101-25df-4768-a83b-25096712c1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:04.227184 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfe101-25df-4768-a83b-25096712c1c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:04.227740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0049f9-2e98-46d4-95e1-801724f1060b None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:04.227920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cfe101-25df-4768-a83b-25096712c1c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:04.228220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cfe101-25df-4768-a83b-25096712c1c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:04.229618 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cfe101-25df-4768-a83b-25096712c1c7 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:04.238118 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 263/509] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Jun 27 18:27:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:04.238936 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfe101-25df-4768-a83b-25096712c1c7 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:04.243414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-6d0049f9-2e98-46d4-95e1-801724f1060b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 27 18:27:04.243414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:04.243414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:04.243414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:04.243414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:04.243414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:04.243414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:04.243414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:04.243414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:04.243414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:04.243414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:04.243414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:04.243414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:04.243414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:04.243414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:04.243414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:04.244316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:04.244316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:04.244316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:04.244316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:04.244316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:04.244316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:04.244316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:04.244316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:04.244316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:04.244316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:04.244316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:04.244316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:04.244316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:04.244316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:04.244316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:04.244316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:04.244316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:04.244316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:04.245199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:04.245199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:04.245199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:04.245199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:04.245199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:04.245199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:04.245199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:04.245199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:04.245199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 27 18:27:04.245199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:04.245199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 247/510] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:27:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:04.246263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0049f9-2e98-46d4-95e1-801724f1060b None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:04.318546 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:04.341468 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfe101-25df-4768-a83b-25096712c1c7 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:04.343974 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:04.347354 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0049f9-2e98-46d4-95e1-801724f1060b None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:04.376032 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34951508-7be0-4438-b089-9625a61dda69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:04.376731 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34951508-7be0-4438-b089-9625a61dda69 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:04.377008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34951508-7be0-4438-b089-9625a61dda69 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:04.377278 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34951508-7be0-4438-b089-9625a61dda69 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:04.384013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c5046-d20c-4ffa-aa2e-42d42b6615f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:04.384692 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c5046-d20c-4ffa-aa2e-42d42b6615f9 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:04.384995 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c5046-d20c-4ffa-aa2e-42d42b6615f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:04.385152 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34951508-7be0-4438-b089-9625a61dda69 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:04.385270 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c5046-d20c-4ffa-aa2e-42d42b6615f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:04.385991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c5046-d20c-4ffa-aa2e-42d42b6615f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:04.386923 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c5046-d20c-4ffa-aa2e-42d42b6615f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:04.387213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34951508-7be0-4438-b089-9625a61dda69 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:04.388454 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c5046-d20c-4ffa-aa2e-42d42b6615f9 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:04.399485 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 248/511] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:04.400259 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c5046-d20c-4ffa-aa2e-42d42b6615f9 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:04.401121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-34951508-7be0-4438-b089-9625a61dda69 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 18:27:04.401121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:04.401121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:04.401121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:04.401121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:04.401121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:04.401121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:04.401121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:04.401121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:04.401121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:04.401121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:04.401121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:04.401121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:04.401121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:04.401121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:04.401121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:04.401995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:04.401995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:04.401995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:04.401995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:04.401995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:04.401995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:04.401995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:04.401995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:04.401995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:04.401995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:04.401995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:04.401995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:04.401995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:04.401995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:04.401995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:04.401995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:04.401995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:04.401995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:04.402905 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:04.402905 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:04.402905 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:04.402905 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:04.402905 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:04.402905 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:04.402905 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:04.402905 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:04.402905 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 18:27:04.402905 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:04.402905 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 264/512] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:27:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:04.403911 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34951508-7be0-4438-b089-9625a61dda69 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:04.510508 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c5046-d20c-4ffa-aa2e-42d42b6615f9 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:04.511956 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34951508-7be0-4438-b089-9625a61dda69 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:04.544412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863dceb-918e-4cd6-9f45-dd1f879d5abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:04.545058 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863dceb-918e-4cd6-9f45-dd1f879d5abe None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:04.545340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863dceb-918e-4cd6-9f45-dd1f879d5abe None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:04.545437 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdbadae-0485-44dc-b873-2ba58f85e3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:04.545614 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863dceb-918e-4cd6-9f45-dd1f879d5abe None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:04.546134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbadae-0485-44dc-b873-2ba58f85e3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:04.546330 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863dceb-918e-4cd6-9f45-dd1f879d5abe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:04.546451 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbadae-0485-44dc-b873-2ba58f85e3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:04.546666 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863dceb-918e-4cd6-9f45-dd1f879d5abe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:04.546753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdbadae-0485-44dc-b873-2ba58f85e3b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:04.548036 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863dceb-918e-4cd6-9f45-dd1f879d5abe None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:04.549321 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:04.557655 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdbadae-0485-44dc-b873-2ba58f85e3b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:04.560142 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdbadae-0485-44dc-b873-2ba58f85e3b8 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:04.560612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 249/513] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:04.561318 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863dceb-918e-4cd6-9f45-dd1f879d5abe None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:04.580105 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:04.581262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-3cdbadae-0485-44dc-b873-2ba58f85e3b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:04.581262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:04.581262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:04.581262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:04.581262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:04.581262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:04.581262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:04.581262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:04.581262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:04.581262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:04.581262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:04.581262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:04.581262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:04.581262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:04.581262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:04.582097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:04.582097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:04.582097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:04.582097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:04.582097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:04.582097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:04.582097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:04.582097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:04.582097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:04.582097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:04.582097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:04.582097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:04.582097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:04.582097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:04.582097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:04.582097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:04.582097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:04.583315 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:04.583315 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:04.583315 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:04.583315 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:04.583315 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 265/514] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:04.584107 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdbadae-0485-44dc-b873-2ba58f85e3b8 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:04.664842 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863dceb-918e-4cd6-9f45-dd1f879d5abe None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:04.684446 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdbadae-0485-44dc-b873-2ba58f85e3b8 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:04.701078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08bd60-7061-4b8b-912f-0ac88478e034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:04.701725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08bd60-7061-4b8b-912f-0ac88478e034 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:04.701999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08bd60-7061-4b8b-912f-0ac88478e034 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:04.702267 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08bd60-7061-4b8b-912f-0ac88478e034 None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:04.703130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08bd60-7061-4b8b-912f-0ac88478e034 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:04.703434 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08bd60-7061-4b8b-912f-0ac88478e034 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:04.704856 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e08bd60-7061-4b8b-912f-0ac88478e034 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:04.717800 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05347e06-b086-4108-a201-44eb3ad194ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:04.718604 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05347e06-b086-4108-a201-44eb3ad194ed None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:04.718887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05347e06-b086-4108-a201-44eb3ad194ed None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:04.719161 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05347e06-b086-4108-a201-44eb3ad194ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:04.721905 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 250/515] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Jun 27 18:27:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:04.722555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08bd60-7061-4b8b-912f-0ac88478e034 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:04.727626 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05347e06-b086-4108-a201-44eb3ad194ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:04.729078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05347e06-b086-4108-a201-44eb3ad194ed None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:04.744830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-05347e06-b086-4108-a201-44eb3ad194ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:04.744830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:04.744830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:04.744830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:04.744830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:04.744830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:04.744830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:04.744830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:04.744830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:04.744830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:04.744830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:04.744830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:04.744830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:04.744830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:04.744830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:04.746365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:04.746365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:04.746365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:04.746365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:04.746365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:04.746365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:04.746365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:04.746365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:04.746365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:04.746365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:04.746365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:04.746365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:04.746365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:04.746365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:04.746365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:04.746365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:04.746365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:04.747242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:04.747242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:04.747242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:04.747242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:04.747242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:04.747242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:04.747242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:04.747242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:04.747242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 266/516] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:04.747712 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05347e06-b086-4108-a201-44eb3ad194ed None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:04.797635 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:04.800991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:04.823814 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08bd60-7061-4b8b-912f-0ac88478e034 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:04.847683 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05347e06-b086-4108-a201-44eb3ad194ed None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:04.857689 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7ff8b-eb65-44b4-8471-c224bae4b4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:04.858346 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7ff8b-eb65-44b4-8471-c224bae4b4e4 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:04.858655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7ff8b-eb65-44b4-8471-c224bae4b4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:04.858928 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7ff8b-eb65-44b4-8471-c224bae4b4e4 None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:04.859749 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7ff8b-eb65-44b4-8471-c224bae4b4e4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:04.860052 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7ff8b-eb65-44b4-8471-c224bae4b4e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:04.861466 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7ff8b-eb65-44b4-8471-c224bae4b4e4 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:04.879020 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 251/517] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Jun 27 18:27:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:04.879801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7ff8b-eb65-44b4-8471-c224bae4b4e4 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:04.887097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae230f-e8e0-4509-9137-c9662b225287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:04.887823 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae230f-e8e0-4509-9137-c9662b225287 None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:04.888105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae230f-e8e0-4509-9137-c9662b225287 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:04.888406 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae230f-e8e0-4509-9137-c9662b225287 None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:04.889109 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae230f-e8e0-4509-9137-c9662b225287 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:04.890585 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae230f-e8e0-4509-9137-c9662b225287 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:04.966238 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 267/518] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Jun 27 18:27:04 2022] POST /identity/v3/users => generated 247 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:27:04.967043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae230f-e8e0-4509-9137-c9662b225287 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:04.978556 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7ff8b-eb65-44b4-8471-c224bae4b4e4 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:05.022536 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df03da-66b1-45c2-b10e-dd58df756c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:05.023562 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df03da-66b1-45c2-b10e-dd58df756c20 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:05.023852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df03da-66b1-45c2-b10e-dd58df756c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:05.024133 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df03da-66b1-45c2-b10e-dd58df756c20 None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:05.025057 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df03da-66b1-45c2-b10e-dd58df756c20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:05.025707 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df03da-66b1-45c2-b10e-dd58df756c20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:05.027633 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df03da-66b1-45c2-b10e-dd58df756c20 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:05.029580 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:05.045959 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 252/519] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:05.045959 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df03da-66b1-45c2-b10e-dd58df756c20 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:05.084719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae230f-e8e0-4509-9137-c9662b225287 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:05.088098 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:05.128603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df638f-9e61-4930-9257-651dcc1f4d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:05.129248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df638f-9e61-4930-9257-651dcc1f4d4c None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:05.129526 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df638f-9e61-4930-9257-651dcc1f4d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:05.129795 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df638f-9e61-4930-9257-651dcc1f4d4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:05.161922 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df03da-66b1-45c2-b10e-dd58df756c20 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:05.203871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3ae44-daae-4832-b4a0-f42ebb580ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:05.204528 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3ae44-daae-4832-b4a0-f42ebb580ccd None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:05.204816 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3ae44-daae-4832-b4a0-f42ebb580ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:05.205090 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3ae44-daae-4832-b4a0-f42ebb580ccd None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:05.205883 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c3ae44-daae-4832-b4a0-f42ebb580ccd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:05.206198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c3ae44-daae-4832-b4a0-f42ebb580ccd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:05.207735 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c3ae44-daae-4832-b4a0-f42ebb580ccd None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:05.211767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df638f-9e61-4930-9257-651dcc1f4d4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:05.213422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df638f-9e61-4930-9257-651dcc1f4d4c None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:05.226735 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 253/520] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:05.227759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3ae44-daae-4832-b4a0-f42ebb580ccd None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:05.284196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:05.321500 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 268/521] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:27:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:27:05.322340 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df638f-9e61-4930-9257-651dcc1f4d4c None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:05.328450 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:05.336078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3ae44-daae-4832-b4a0-f42ebb580ccd None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:05.431301 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d357b3-6f96-4c1b-8974-83c0891c6e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:05.431951 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d357b3-6f96-4c1b-8974-83c0891c6e3c None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:05.432230 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d357b3-6f96-4c1b-8974-83c0891c6e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:05.432508 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d357b3-6f96-4c1b-8974-83c0891c6e3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:05.507704 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d357b3-6f96-4c1b-8974-83c0891c6e3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:05.509083 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d357b3-6f96-4c1b-8974-83c0891c6e3c None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:05.509279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df638f-9e61-4930-9257-651dcc1f4d4c None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:05.550773 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c762f89-70c7-4231-bc58-939a8c734afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:05.552160 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c762f89-70c7-4231-bc58-939a8c734afa None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:05.552569 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c762f89-70c7-4231-bc58-939a8c734afa None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:05.552946 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c762f89-70c7-4231-bc58-939a8c734afa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:05.563614 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c762f89-70c7-4231-bc58-939a8c734afa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:05.565932 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:05.566036 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c762f89-70c7-4231-bc58-939a8c734afa None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:05.586260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-9c762f89-70c7-4231-bc58-939a8c734afa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 18:27:05.586260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:05.586260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:05.586260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:05.586260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:05.586260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:05.586260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:05.586260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:05.586260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:05.586260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:05.586260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:05.586260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:05.586260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:05.586260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:05.586260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:05.586260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:05.593881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:05.593881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:05.593881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:05.593881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:05.593881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:05.593881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:05.593881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:05.593881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:05.593881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:05.593881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:05.593881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:05.593881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:05.593881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:05.593881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:05.593881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:05.593881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:05.593881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:05.593881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:05.595046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:05.595046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:05.595046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:05.595046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:05.595046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:05.595046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:05.595046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:05.595046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:05.595046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 18:27:05.595046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:05.595046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 269/522] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:27:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:05.595046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c762f89-70c7-4231-bc58-939a8c734afa None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:05.617193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 254/523] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:27:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:27:05.618237 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d357b3-6f96-4c1b-8974-83c0891c6e3c None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:05.663236 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:05.701883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c762f89-70c7-4231-bc58-939a8c734afa None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:05.725308 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d357b3-6f96-4c1b-8974-83c0891c6e3c None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:05.745434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae97918-3c5b-4747-9a6a-8f2e18b9e7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:05.745434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae97918-3c5b-4747-9a6a-8f2e18b9e7a5 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:05.745434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae97918-3c5b-4747-9a6a-8f2e18b9e7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:05.745434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae97918-3c5b-4747-9a6a-8f2e18b9e7a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:05.746004 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae97918-3c5b-4747-9a6a-8f2e18b9e7a5 None admin] RBAC: Adding query filter params (role.id=c4bc6e72b9d647c5b2485d52d55f7dd4, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=1dddd4efb44c4152bb4151f7715f2223) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:05.746004 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae97918-3c5b-4747-9a6a-8f2e18b9e7a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:05.747409 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae97918-3c5b-4747-9a6a-8f2e18b9e7a5 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:05.753230 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 270/524] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:27:05 2022] GET /identity/v3/role_assignments?user.id=1dddd4efb44c4152bb4151f7715f2223&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=c4bc6e72b9d647c5b2485d52d55f7dd4 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:05.755528 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae97918-3c5b-4747-9a6a-8f2e18b9e7a5 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:05.766148 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250315b-ebcb-41b9-862d-308074e677b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:05.766905 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250315b-ebcb-41b9-862d-308074e677b7 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:05.767198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250315b-ebcb-41b9-862d-308074e677b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:05.767542 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250315b-ebcb-41b9-862d-308074e677b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:05.780923 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f250315b-ebcb-41b9-862d-308074e677b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:05.782517 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f250315b-ebcb-41b9-862d-308074e677b7 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:05.803171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-f250315b-ebcb-41b9-862d-308074e677b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:05.803171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:05.803171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:05.803171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:05.803171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:05.803171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:05.803171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:05.803171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:05.803171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:05.803171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:05.803171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:05.803171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:05.803171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:05.803171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:05.803171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:05.804131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:05.804131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:05.804131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:05.804131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:05.804131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:05.804131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:05.804131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:05.804131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:05.804131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:05.804131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:05.804131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:05.804131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:05.804131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:05.804131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:05.804131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:05.804131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:05.804131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:05.804968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:05.804968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:05.804968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:05.804968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:05.804968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 255/525] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:05.810565 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250315b-ebcb-41b9-862d-308074e677b7 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:05.820831 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:05.888189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae97918-3c5b-4747-9a6a-8f2e18b9e7a5 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:05.918401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:05.929645 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ea8cd-745b-45bc-abfd-a69084f6d645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:05.930473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ea8cd-745b-45bc-abfd-a69084f6d645 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:05.930782 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250315b-ebcb-41b9-862d-308074e677b7 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:05.930928 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ea8cd-745b-45bc-abfd-a69084f6d645 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:05.931305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ea8cd-745b-45bc-abfd-a69084f6d645 None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:05.932248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ea8cd-745b-45bc-abfd-a69084f6d645 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:05.932659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ea8cd-745b-45bc-abfd-a69084f6d645 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:05.936049 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ea8cd-745b-45bc-abfd-a69084f6d645 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:05.969263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7142df61-b0a0-4141-847b-472016758290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:05.970324 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142df61-b0a0-4141-847b-472016758290 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:05.970772 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142df61-b0a0-4141-847b-472016758290 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:05.971203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142df61-b0a0-4141-847b-472016758290 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:05.982501 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7142df61-b0a0-4141-847b-472016758290 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:05.984330 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7142df61-b0a0-4141-847b-472016758290 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:05.987258 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 271/526] 10.222.0.15 () {56 vars in 1139 bytes} [Mon Jun 27 18:27:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:05.988732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ea8cd-745b-45bc-abfd-a69084f6d645 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:06.005819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-7142df61-b0a0-4141-847b-472016758290 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:06.005819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:06.005819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:06.005819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:06.005819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:06.005819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:06.005819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:06.005819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:06.005819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:06.005819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:06.005819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:06.005819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:06.005819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:06.005819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:06.005819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:06.006784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:06.006784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:06.006784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:06.006784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:06.006784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:06.006784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:06.006784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:06.006784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:06.006784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:06.006784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:06.006784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:06.006784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:06.006784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:06.006784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:06.006784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:06.006784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:06.006784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:06.007624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:06.007624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:06.007624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:06.007624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:06.010065 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 256/527] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:06.011348 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7142df61-b0a0-4141-847b-472016758290 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:06.077798 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:06.124503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ea8cd-745b-45bc-abfd-a69084f6d645 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:06.125187 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7142df61-b0a0-4141-847b-472016758290 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:06.163111 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1fc85-0592-49b8-ad6e-e95c53ff2861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:06.163877 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1fc85-0592-49b8-ad6e-e95c53ff2861 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:06.164226 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1fc85-0592-49b8-ad6e-e95c53ff2861 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:06.164565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1fc85-0592-49b8-ad6e-e95c53ff2861 None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:06.165396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d1fc85-0592-49b8-ad6e-e95c53ff2861 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:06.165775 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d1fc85-0592-49b8-ad6e-e95c53ff2861 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:06.166531 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59cec5-e600-45d8-a334-11c932d89044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:06.167321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59cec5-e600-45d8-a334-11c932d89044 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:06.167967 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d1fc85-0592-49b8-ad6e-e95c53ff2861 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:06.168926 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59cec5-e600-45d8-a334-11c932d89044 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:06.169448 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59cec5-e600-45d8-a334-11c932d89044 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:06.180481 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 272/528] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:06.181248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1fc85-0592-49b8-ad6e-e95c53ff2861 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:06.233083 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:06.249295 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59cec5-e600-45d8-a334-11c932d89044 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:06.250806 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59cec5-e600-45d8-a334-11c932d89044 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:06.301243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1fc85-0592-49b8-ad6e-e95c53ff2861 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:06.327595 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:06.338001 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e083e2d-133c-4929-a55e-d4a024013df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:06.338760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e083e2d-133c-4929-a55e-d4a024013df8 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:06.339142 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e083e2d-133c-4929-a55e-d4a024013df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:06.340628 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e083e2d-133c-4929-a55e-d4a024013df8 None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:06.341432 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e083e2d-133c-4929-a55e-d4a024013df8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:06.341808 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e083e2d-133c-4929-a55e-d4a024013df8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:06.343283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e083e2d-133c-4929-a55e-d4a024013df8 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:06.356422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 273/529] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:06.357658 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e083e2d-133c-4929-a55e-d4a024013df8 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:06.382506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 257/530] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:27:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:27:06.383347 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59cec5-e600-45d8-a334-11c932d89044 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:06.477240 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e083e2d-133c-4929-a55e-d4a024013df8 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:06.486028 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:06.510527 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59cec5-e600-45d8-a334-11c932d89044 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:06.514526 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b32da4-0c2a-48f4-872b-ef12c6446732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:06.515534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b32da4-0c2a-48f4-872b-ef12c6446732 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:06.515937 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b32da4-0c2a-48f4-872b-ef12c6446732 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:06.516308 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b32da4-0c2a-48f4-872b-ef12c6446732 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:06.523600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b32da4-0c2a-48f4-872b-ef12c6446732 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:06.525468 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b32da4-0c2a-48f4-872b-ef12c6446732 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:06.541559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-19b32da4-0c2a-48f4-872b-ef12c6446732 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:06.541559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:06.541559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:06.541559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:06.541559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:06.541559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:06.541559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:06.541559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:06.541559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:06.541559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:06.541559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:06.541559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:06.541559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:06.541559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:06.541559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:06.542663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:06.542663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:06.542663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:06.542663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:06.542663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:06.542663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:06.542663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:06.542663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:06.542663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:06.542663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:06.542663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:06.542663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:06.542663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:06.542663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:06.542663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:06.542663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:06.542663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:06.543596 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:06.543596 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:06.543596 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:06.543596 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:06.543596 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:06.543596 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:06.543596 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:06.543596 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:06.546165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 274/531] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:06.546339 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b32da4-0c2a-48f4-872b-ef12c6446732 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:06.548044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7f782-66fb-4f65-9c45-cafee997489c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:06.548828 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7f782-66fb-4f65-9c45-cafee997489c None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:06.549217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7f782-66fb-4f65-9c45-cafee997489c None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:06.549499 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7f782-66fb-4f65-9c45-cafee997489c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:06.557110 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7f782-66fb-4f65-9c45-cafee997489c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:06.559365 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7f782-66fb-4f65-9c45-cafee997489c None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:06.574852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-adb7f782-66fb-4f65-9c45-cafee997489c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:06.574852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:06.574852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:06.574852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:06.574852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:06.574852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:06.574852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:06.574852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:06.574852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:06.574852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:06.574852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:06.574852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:06.574852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:06.574852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:06.574852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:06.575864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:06.575864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:06.575864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:06.575864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:06.575864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:06.575864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:06.575864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:06.575864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:06.575864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:06.575864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:06.575864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:06.575864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:06.575864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:06.575864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:06.575864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:06.575864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:06.575864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:06.577072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:06.577072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:06.577072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:06.577072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:06.577072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:06.577072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:06.577072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:06.577072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:06.577072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 258/532] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:06.577716 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7f782-66fb-4f65-9c45-cafee997489c None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:06.598502 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:06.657990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b32da4-0c2a-48f4-872b-ef12c6446732 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:06.688176 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7f782-66fb-4f65-9c45-cafee997489c None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:06.694031 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a5021-5421-4a00-8650-6156248de5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:06.694960 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a5021-5421-4a00-8650-6156248de5a8 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:06.695342 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a5021-5421-4a00-8650-6156248de5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:06.695706 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a5021-5421-4a00-8650-6156248de5a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:06.707457 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a5021-5421-4a00-8650-6156248de5a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:06.709042 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a5021-5421-4a00-8650-6156248de5a8 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:06.726852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04ec0f-f7f9-4ea4-8f37-217523d0a93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:06.727345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-a61a5021-5421-4a00-8650-6156248de5a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:06.727345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:06.727345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:06.727345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:06.727345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:06.727345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:06.727345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:06.727345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:06.727345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:06.727345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:06.727345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:06.727345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:06.727345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:06.727345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:06.727345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:06.728324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:06.728324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:06.728324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:06.728324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:06.728324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:06.728324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:06.728324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:06.728324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:06.728324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:06.728324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:06.728324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:06.728324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:06.728324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:06.728324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:06.728324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:06.728324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:06.728324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:06.729362 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:06.729362 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:06.729362 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:06.729362 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:06.730471 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04ec0f-f7f9-4ea4-8f37-217523d0a93b None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:06.730932 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04ec0f-f7f9-4ea4-8f37-217523d0a93b None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:06.731384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04ec0f-f7f9-4ea4-8f37-217523d0a93b None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:06.736011 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 275/533] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:06.736011 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a5021-5421-4a00-8650-6156248de5a8 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:06.737252 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04ec0f-f7f9-4ea4-8f37-217523d0a93b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:06.737770 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04ec0f-f7f9-4ea4-8f37-217523d0a93b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:06.739423 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04ec0f-f7f9-4ea4-8f37-217523d0a93b None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:06.753827 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 259/534] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:06.755057 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04ec0f-f7f9-4ea4-8f37-217523d0a93b None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:06.756386 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:06.843283 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:06.846522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a5021-5421-4a00-8650-6156248de5a8 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:06.867374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04ec0f-f7f9-4ea4-8f37-217523d0a93b None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:06.884443 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88b6c7-b6d8-49a0-b481-8a68640f0538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:06.885209 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88b6c7-b6d8-49a0-b481-8a68640f0538 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:06.885555 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88b6c7-b6d8-49a0-b481-8a68640f0538 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:06.885935 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88b6c7-b6d8-49a0-b481-8a68640f0538 None admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:06.888025 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db88b6c7-b6d8-49a0-b481-8a68640f0538 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:06.888524 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db88b6c7-b6d8-49a0-b481-8a68640f0538 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:06.889940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db88b6c7-b6d8-49a0-b481-8a68640f0538 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:06.907645 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899dfa1-d84c-4385-b449-f752f9d2420b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:06.908657 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899dfa1-d84c-4385-b449-f752f9d2420b None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:06.909138 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899dfa1-d84c-4385-b449-f752f9d2420b None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:06.909590 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899dfa1-d84c-4385-b449-f752f9d2420b None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:06.910509 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 276/535] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:06.910741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8899dfa1-d84c-4385-b449-f752f9d2420b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:06.911281 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8899dfa1-d84c-4385-b449-f752f9d2420b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:06.911649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88b6c7-b6d8-49a0-b481-8a68640f0538 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:06.913119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8899dfa1-d84c-4385-b449-f752f9d2420b None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:06.932249 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 260/536] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:06.933454 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899dfa1-d84c-4385-b449-f752f9d2420b None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:06.995333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:07.018769 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88b6c7-b6d8-49a0-b481-8a68640f0538 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:07.045979 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899dfa1-d84c-4385-b449-f752f9d2420b None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:07.055213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627e8c3-ac74-4462-85d0-06e58148299d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:07.055970 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627e8c3-ac74-4462-85d0-06e58148299d None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:07.056319 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627e8c3-ac74-4462-85d0-06e58148299d None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:07.056664 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627e8c3-ac74-4462-85d0-06e58148299d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:07.057560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627e8c3-ac74-4462-85d0-06e58148299d None admin] RBAC: Adding query filter params (role.id=c4bc6e72b9d647c5b2485d52d55f7dd4, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=75e75e5ae3ca45d0b503f6e52f399558) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:07.057937 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627e8c3-ac74-4462-85d0-06e58148299d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:07.060228 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627e8c3-ac74-4462-85d0-06e58148299d None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:07.066437 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 277/537] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:27:06 2022] GET /identity/v3/role_assignments?user.id=75e75e5ae3ca45d0b503f6e52f399558&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=c4bc6e72b9d647c5b2485d52d55f7dd4 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:07.067311 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627e8c3-ac74-4462-85d0-06e58148299d None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:07.087098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee40cd-7f1d-43fb-b553-88c7cf8272d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:07.088122 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee40cd-7f1d-43fb-b553-88c7cf8272d3 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:07.088661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee40cd-7f1d-43fb-b553-88c7cf8272d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:07.089114 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee40cd-7f1d-43fb-b553-88c7cf8272d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:07.090346 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee40cd-7f1d-43fb-b553-88c7cf8272d3 None admin] RBAC: Adding query filter params (role.id=c4bc6e72b9d647c5b2485d52d55f7dd4, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=c4f6f8e0a0cb4c9a9dd0d99fc0022235) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:07.090866 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee40cd-7f1d-43fb-b553-88c7cf8272d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:07.092498 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee40cd-7f1d-43fb-b553-88c7cf8272d3 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:07.099608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 261/538] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:27:06 2022] GET /identity/v3/role_assignments?user.id=c4f6f8e0a0cb4c9a9dd0d99fc0022235&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=c4bc6e72b9d647c5b2485d52d55f7dd4 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:07.127126 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f9273-067b-4f6c-a665-ba612c42991a None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:07.127759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f9273-067b-4f6c-a665-ba612c42991a None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:07.128238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f9273-067b-4f6c-a665-ba612c42991a None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:07.130085 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 262/539] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:07.136082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1cded-fcc4-4b1b-9e62-086e4c15eb5e None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:07.137257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1cded-fcc4-4b1b-9e62-086e4c15eb5e None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:07.137763 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1cded-fcc4-4b1b-9e62-086e4c15eb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:07.157560 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:07.193058 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-3cc1cded-fcc4-4b1b-9e62-086e4c15eb5e None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:07.194907 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-3cc1cded-fcc4-4b1b-9e62-086e4c15eb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:07.199216 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627e8c3-ac74-4462-85d0-06e58148299d None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:07.253347 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a209e382-acda-44c6-b4b8-4224217620d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:07.255215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209e382-acda-44c6-b4b8-4224217620d8 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:07.257304 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209e382-acda-44c6-b4b8-4224217620d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:07.257823 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209e382-acda-44c6-b4b8-4224217620d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:07.275640 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a209e382-acda-44c6-b4b8-4224217620d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:07.278435 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a209e382-acda-44c6-b4b8-4224217620d8 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:07.295187 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-a209e382-acda-44c6-b4b8-4224217620d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:07.295187 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:07.295187 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:07.295187 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:07.295187 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:07.295187 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:07.295187 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:07.295187 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:07.295187 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:07.295187 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:07.295187 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:07.295187 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:07.295187 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:07.295187 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:07.295187 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:07.296213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:07.296213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:07.296213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:07.296213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:07.296213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:07.296213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:07.296213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:07.296213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:07.296213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:07.296213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:07.296213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:07.296213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:07.296213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:07.296213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:07.296213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:07.296213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:07.296213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:07.297069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:07.297069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:07.297069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:07.297069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:07.297069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:07.297069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:07.297069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:07.297069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:07.297853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 278/540] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:07.306252 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a209e382-acda-44c6-b4b8-4224217620d8 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:07.308838 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 263/541] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:27:07.312323 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:07.334759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1cded-fcc4-4b1b-9e62-086e4c15eb5e None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:07.451367 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1cded-fcc4-4b1b-9e62-086e4c15eb5e None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:07.475888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a209e382-acda-44c6-b4b8-4224217620d8 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:07.485601 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:07.499825 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c15f70-fbc5-490a-a657-8de21337ca89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:07.501036 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c15f70-fbc5-490a-a657-8de21337ca89 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:07.501506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c15f70-fbc5-490a-a657-8de21337ca89 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:07.502092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c15f70-fbc5-490a-a657-8de21337ca89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:07.524830 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c15f70-fbc5-490a-a657-8de21337ca89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:07.525303 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ff0a5-d442-4eb3-abc4-a614305a119a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:07.526055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ff0a5-d442-4eb3-abc4-a614305a119a None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:07.526429 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ff0a5-d442-4eb3-abc4-a614305a119a None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:07.527352 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ff0a5-d442-4eb3-abc4-a614305a119a None admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:07.528335 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ff0a5-d442-4eb3-abc4-a614305a119a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:07.528715 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ff0a5-d442-4eb3-abc4-a614305a119a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:07.530247 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ff0a5-d442-4eb3-abc4-a614305a119a None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:07.543608 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 279/542] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:07.547208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-75c15f70-fbc5-490a-a657-8de21337ca89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:07.547208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:07.547208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:07.547208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:07.547208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:07.547208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:07.547208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:07.547208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:07.547208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:07.547208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:07.547208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:07.547208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:07.547208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:07.547208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:07.547208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:07.547208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:07.548070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:07.548070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:07.548070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:07.548070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:07.548070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:07.548070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:07.548070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:07.548070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:07.548070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:07.548070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:07.548070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:07.548070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:07.548070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:07.548070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:07.548070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:07.548070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:07.548867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:07.548867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:07.548867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:07.548867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:07.548867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:07.548867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:07.548867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:07.548867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:07.548867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:07.548867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:07.548867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:07.548867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:07.549464 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ff0a5-d442-4eb3-abc4-a614305a119a None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:07.551454 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 264/543] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:27:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:07.575225 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c15f70-fbc5-490a-a657-8de21337ca89 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:07.628508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:07.666322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ff0a5-d442-4eb3-abc4-a614305a119a None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:07.729336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f57d8-3fa1-45f8-b855-1cb5d054e54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:07.731235 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f57d8-3fa1-45f8-b855-1cb5d054e54c None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:07.731987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f57d8-3fa1-45f8-b855-1cb5d054e54c None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:07.732454 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f57d8-3fa1-45f8-b855-1cb5d054e54c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:07.733466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f57d8-3fa1-45f8-b855-1cb5d054e54c None admin] RBAC: Adding query filter params (role.id=c4bc6e72b9d647c5b2485d52d55f7dd4, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=dbe621916dbe4f1a8686097fe2bdf762) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:07.734919 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f57d8-3fa1-45f8-b855-1cb5d054e54c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:07.739050 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f57d8-3fa1-45f8-b855-1cb5d054e54c None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:07.756574 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 280/544] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:27:07 2022] GET /identity/v3/role_assignments?user.id=dbe621916dbe4f1a8686097fe2bdf762&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=c4bc6e72b9d647c5b2485d52d55f7dd4 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:07.784336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c15f70-fbc5-490a-a657-8de21337ca89 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:07.828274 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3bf0cd-2c8f-43ae-9e85-d9365fa97505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:07.828960 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bf0cd-2c8f-43ae-9e85-d9365fa97505 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:07.829246 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bf0cd-2c8f-43ae-9e85-d9365fa97505 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:07.829525 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bf0cd-2c8f-43ae-9e85-d9365fa97505 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:07.830283 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3bf0cd-2c8f-43ae-9e85-d9365fa97505 None admin] RBAC: Adding query filter params (name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:07.830623 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3bf0cd-2c8f-43ae-9e85-d9365fa97505 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:07.832010 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3bf0cd-2c8f-43ae-9e85-d9365fa97505 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:07.841577 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 265/545] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Jun 27 18:27:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:07.847083 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f57d8-3fa1-45f8-b855-1cb5d054e54c None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:07.944655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4625391-4a63-4247-a80b-4eca52894b3f None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:07.945003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4625391-4a63-4247-a80b-4eca52894b3f None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:07.945209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4625391-4a63-4247-a80b-4eca52894b3f None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:07.946633 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 266/546] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:27:07.952304 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaffeba-e270-4dc8-8d16-3e6a405fc2ed None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:07.953022 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaffeba-e270-4dc8-8d16-3e6a405fc2ed None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:07.953611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaffeba-e270-4dc8-8d16-3e6a405fc2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:07.972541 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-fdaffeba-e270-4dc8-8d16-3e6a405fc2ed None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:07.974071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-fdaffeba-e270-4dc8-8d16-3e6a405fc2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:08.014484 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f57d8-3fa1-45f8-b855-1cb5d054e54c None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:08.070233 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:08.076129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:08.084834 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df2b36-a683-4a47-9237-2933a4ab59df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:08.085995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df2b36-a683-4a47-9237-2933a4ab59df None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:08.087259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df2b36-a683-4a47-9237-2933a4ab59df None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:08.091475 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 267/547] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:27:08.091853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df2b36-a683-4a47-9237-2933a4ab59df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:08.103072 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df2b36-a683-4a47-9237-2933a4ab59df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:08.105010 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df2b36-a683-4a47-9237-2933a4ab59df None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:08.124280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-f6df2b36-a683-4a47-9237-2933a4ab59df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:08.124280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:08.124280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:08.124280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:08.124280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:08.124280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:08.124280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:08.124280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:08.124280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:08.124280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:08.124280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:08.124280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:08.124280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:08.124280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:08.124280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:08.125248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:08.125248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:08.125248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:08.125248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:08.125248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:08.125248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:08.125248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:08.125248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:08.125248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:08.125248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:08.125248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:08.125248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:08.125248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:08.125248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:08.125248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:08.125248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:08.125248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:08.126589 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:08.126589 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:08.126589 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:08.126589 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:08.128546 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 281/548] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:08.131946 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaffeba-e270-4dc8-8d16-3e6a405fc2ed None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:08.143440 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df2b36-a683-4a47-9237-2933a4ab59df None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:08.303622 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df2b36-a683-4a47-9237-2933a4ab59df None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:08.343798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007d6be-656d-4e9d-956a-5f579a29a7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:08.344464 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007d6be-656d-4e9d-956a-5f579a29a7bc None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:08.344744 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007d6be-656d-4e9d-956a-5f579a29a7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:08.345016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007d6be-656d-4e9d-956a-5f579a29a7bc None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:08.345747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007d6be-656d-4e9d-956a-5f579a29a7bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:08.346053 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007d6be-656d-4e9d-956a-5f579a29a7bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:08.347457 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007d6be-656d-4e9d-956a-5f579a29a7bc None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:08.351013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaffeba-e270-4dc8-8d16-3e6a405fc2ed None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:08.371976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 282/549] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:08.378280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007d6be-656d-4e9d-956a-5f579a29a7bc None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:08.423564 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03331fa5-0c8a-4176-a134-9ae161095639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:08.424385 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03331fa5-0c8a-4176-a134-9ae161095639 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:08.424675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03331fa5-0c8a-4176-a134-9ae161095639 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:08.424949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03331fa5-0c8a-4176-a134-9ae161095639 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:08.448536 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03331fa5-0c8a-4176-a134-9ae161095639 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:08.460791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-03331fa5-0c8a-4176-a134-9ae161095639 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:08.460791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:08.460791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:08.460791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:08.460791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:08.460791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:08.460791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:08.460791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:08.460791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:08.460791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:08.460791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:08.460791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:08.460791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:08.460791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:08.460791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:08.460791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:08.461826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:08.461826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:08.461826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:08.461826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:08.461826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:08.461826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:08.461826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:08.461826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:08.461826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:08.461826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:08.461826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:08.461826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:08.461826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:08.461826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:08.461826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:08.461826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:08.462656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:08.462656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:08.462656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:08.462656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:08.462656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:08.462656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:08.462656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:08.462656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:08.462656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:08.462656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:08.462656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:08.462656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:08.466777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:08.467699 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 268/550] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:27:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:08.475548 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03331fa5-0c8a-4176-a134-9ae161095639 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:08.541525 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007d6be-656d-4e9d-956a-5f579a29a7bc None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:08.583961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a608a-c714-47fb-9a2d-472561b32c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:08.587046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a608a-c714-47fb-9a2d-472561b32c3e None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:08.587593 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a608a-c714-47fb-9a2d-472561b32c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:08.588072 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a608a-c714-47fb-9a2d-472561b32c3e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:08.602058 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a608a-c714-47fb-9a2d-472561b32c3e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:08.604325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a608a-c714-47fb-9a2d-472561b32c3e None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:08.616037 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:08.637405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-2b0a608a-c714-47fb-9a2d-472561b32c3e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 18:27:08.637405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:08.637405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:08.637405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:08.637405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:08.637405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:08.637405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:08.637405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:08.637405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:08.637405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:08.637405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:08.637405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:08.637405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:08.637405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:08.637405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:08.637405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:08.638561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:08.638561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:08.638561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:08.638561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:08.638561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:08.638561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:08.638561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:08.638561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:08.638561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:08.638561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:08.638561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:08.638561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:08.638561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:08.638561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:08.638561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:08.638561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:08.638561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:08.638561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:08.639488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:08.639488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:08.639488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:08.639488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:08.639488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:08.639488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:08.639488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:08.639488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:08.639488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 18:27:08.639488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:08.639488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 283/551] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:27:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:08.644154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a608a-c714-47fb-9a2d-472561b32c3e None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:08.692872 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03331fa5-0c8a-4176-a134-9ae161095639 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:08.735586 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83a883-52ae-4f50-9e96-73db1e0c25df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:08.736243 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83a883-52ae-4f50-9e96-73db1e0c25df None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:08.736524 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83a883-52ae-4f50-9e96-73db1e0c25df None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:08.736798 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83a883-52ae-4f50-9e96-73db1e0c25df None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:08.737546 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83a883-52ae-4f50-9e96-73db1e0c25df None admin] RBAC: Adding query filter params (name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:08.737851 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83a883-52ae-4f50-9e96-73db1e0c25df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:08.739251 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee83a883-52ae-4f50-9e96-73db1e0c25df None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:08.759044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 269/552] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Jun 27 18:27:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:08.763072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83a883-52ae-4f50-9e96-73db1e0c25df None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:08.883844 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83a883-52ae-4f50-9e96-73db1e0c25df None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:08.893043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:08.895329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:08.906354 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a608a-c714-47fb-9a2d-472561b32c3e None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:08.929543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510d6c1-8a48-4b49-b6b6-cbe301417145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:08.930567 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510d6c1-8a48-4b49-b6b6-cbe301417145 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:08.932006 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510d6c1-8a48-4b49-b6b6-cbe301417145 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:08.932283 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510d6c1-8a48-4b49-b6b6-cbe301417145 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:08.953385 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510d6c1-8a48-4b49-b6b6-cbe301417145 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:08.955732 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510d6c1-8a48-4b49-b6b6-cbe301417145 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:08.980741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-4510d6c1-8a48-4b49-b6b6-cbe301417145 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:08.980741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:08.980741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:08.980741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:08.980741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:08.980741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:08.980741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:08.980741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:08.980741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:08.980741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:08.980741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:08.980741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:08.980741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:08.980741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:08.980741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:08.981936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:08.981936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:08.981936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:08.981936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:08.981936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:08.981936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:08.981936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:08.981936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:08.981936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:08.981936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:08.981936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:08.981936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:08.981936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:08.981936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:08.981936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:08.981936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:08.981936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:08.982817 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:08.982817 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:08.982817 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:08.982817 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:08.985086 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 270/553] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:27:08.993032 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e803e2-30f7-4210-be96-391168ddd09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:08.993032 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e803e2-30f7-4210-be96-391168ddd09a None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:08.993032 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e803e2-30f7-4210-be96-391168ddd09a None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:08.993032 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e803e2-30f7-4210-be96-391168ddd09a None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:08.993386 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e803e2-30f7-4210-be96-391168ddd09a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:08.993386 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e803e2-30f7-4210-be96-391168ddd09a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:08.993386 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e803e2-30f7-4210-be96-391168ddd09a None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:08.998876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510d6c1-8a48-4b49-b6b6-cbe301417145 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:09.023492 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 284/554] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Jun 27 18:27:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:09.030136 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e803e2-30f7-4210-be96-391168ddd09a None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:09.172016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e803e2-30f7-4210-be96-391168ddd09a None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:09.215026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8a5f7-f54b-42fb-acb6-5705033418ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:09.215675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8a5f7-f54b-42fb-acb6-5705033418ea None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:09.215956 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8a5f7-f54b-42fb-acb6-5705033418ea None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:09.216232 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8a5f7-f54b-42fb-acb6-5705033418ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:09.236534 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510d6c1-8a48-4b49-b6b6-cbe301417145 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:09.246119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:09.300907 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b8f08-39c6-4720-9850-44d6ea5583f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:09.302089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b8f08-39c6-4720-9850-44d6ea5583f1 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:09.302759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b8f08-39c6-4720-9850-44d6ea5583f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:09.304354 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b8f08-39c6-4720-9850-44d6ea5583f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:09.305305 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918b8f08-39c6-4720-9850-44d6ea5583f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:09.306604 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918b8f08-39c6-4720-9850-44d6ea5583f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:09.308706 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918b8f08-39c6-4720-9850-44d6ea5583f1 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:09.330347 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 271/555] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:09.331503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed8a5f7-f54b-42fb-acb6-5705033418ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:09.333148 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed8a5f7-f54b-42fb-acb6-5705033418ea None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:09.336072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b8f08-39c6-4720-9850-44d6ea5583f1 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:09.368570 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:09.500515 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 285/556] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:27:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 18:27:09.501874 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1209fcc-9824-4dfb-8471-22187209a479 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:09.502173 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1209fcc-9824-4dfb-8471-22187209a479 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:09.502471 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1209fcc-9824-4dfb-8471-22187209a479 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:09.507842 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 286/557] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:09.508614 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1209fcc-9824-4dfb-8471-22187209a479 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:09.508995 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b8f08-39c6-4720-9850-44d6ea5583f1 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:09.556054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0c806-4f4f-40af-bca5-b3b8f51d3d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:09.556841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0c806-4f4f-40af-bca5-b3b8f51d3d26 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:09.557121 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0c806-4f4f-40af-bca5-b3b8f51d3d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:09.557392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0c806-4f4f-40af-bca5-b3b8f51d3d26 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:09.565814 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0c806-4f4f-40af-bca5-b3b8f51d3d26 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:09.567325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0c806-4f4f-40af-bca5-b3b8f51d3d26 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:09.571745 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:09.581811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-84f0c806-4f4f-40af-bca5-b3b8f51d3d26 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 18:27:09.581811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:09.581811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:09.581811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:09.581811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:09.581811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:09.581811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:09.581811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:09.581811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:09.581811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:09.581811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:09.581811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:09.581811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:09.581811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:09.581811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:09.581811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:09.583338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:09.583338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:09.583338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:09.583338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:09.583338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:09.583338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:09.583338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:09.583338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:09.583338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:09.583338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:09.583338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:09.583338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:09.583338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:09.583338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:09.583338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:09.583338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:09.583338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:09.583338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:09.584266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:09.584266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:09.584266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:09.584266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:09.584266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:09.584266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:09.584266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:09.584266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:09.584266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 18:27:09.584266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:09.586579 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 272/558] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Jun 27 18:27:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:09.588002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502aae8-c9cd-4b58-abd0-0468a68a3fa5 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:09.592098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502aae8-c9cd-4b58-abd0-0468a68a3fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:09.592098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502aae8-c9cd-4b58-abd0-0468a68a3fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:09.607206 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-2502aae8-c9cd-4b58-abd0-0468a68a3fa5 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:09.608376 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-2502aae8-c9cd-4b58-abd0-0468a68a3fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:09.627952 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1209fcc-9824-4dfb-8471-22187209a479 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:09.663624 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64557675-2c88-4912-a06b-a3c9479e6e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:09.664414 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64557675-2c88-4912-a06b-a3c9479e6e1b None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:09.664699 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64557675-2c88-4912-a06b-a3c9479e6e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:09.664972 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64557675-2c88-4912-a06b-a3c9479e6e1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:09.676607 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64557675-2c88-4912-a06b-a3c9479e6e1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:09.678055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64557675-2c88-4912-a06b-a3c9479e6e1b None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:09.685453 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 273/559] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:27:09.685453 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c70b5f-5e77-4270-8b49-f756c2d4f42b None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:09.685453 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c70b5f-5e77-4270-8b49-f756c2d4f42b None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:09.685453 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c70b5f-5e77-4270-8b49-f756c2d4f42b None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:09.688220 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 274/560] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:09.688220 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c70b5f-5e77-4270-8b49-f756c2d4f42b None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:09.701480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-64557675-2c88-4912-a06b-a3c9479e6e1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:09.701480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:09.701480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:09.701480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:09.701480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:09.701480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:09.701480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:09.701480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:09.701480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:09.701480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:09.701480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:09.701480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:09.701480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:09.701480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:09.701480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:09.702434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:09.702434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:09.702434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:09.702434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:09.702434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:09.702434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:09.702434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:09.702434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:09.702434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:09.702434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:09.702434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:09.702434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:09.702434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:09.702434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:09.702434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:09.702434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:09.702434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:09.703911 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:09.703911 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:09.703911 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:09.703911 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:09.703911 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 287/561] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:09.706344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17890024-d53b-4e70-b8db-b52a8e9197de None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:09.706344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17890024-d53b-4e70-b8db-b52a8e9197de None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:09.706344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17890024-d53b-4e70-b8db-b52a8e9197de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:09.707403 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:09.723447 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-17890024-d53b-4e70-b8db-b52a8e9197de None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:09.724747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-17890024-d53b-4e70-b8db-b52a8e9197de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:09.810498 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 288/562] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:27:09.810498 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17890024-d53b-4e70-b8db-b52a8e9197de None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:09.823073 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c70b5f-5e77-4270-8b49-f756c2d4f42b None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:09.842830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:09.863268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f1267b-c65a-49ae-8128-f9386f674007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:09.864050 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1267b-c65a-49ae-8128-f9386f674007 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:09.864338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1267b-c65a-49ae-8128-f9386f674007 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:09.864614 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1267b-c65a-49ae-8128-f9386f674007 None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:09.865455 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f1267b-c65a-49ae-8128-f9386f674007 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:09.865754 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f1267b-c65a-49ae-8128-f9386f674007 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:09.867223 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f1267b-c65a-49ae-8128-f9386f674007 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:09.887919 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 275/563] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Jun 27 18:27:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:09.888762 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f1267b-c65a-49ae-8128-f9386f674007 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:09.913390 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17890024-d53b-4e70-b8db-b52a8e9197de None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:09.948453 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddcbc08-dbbf-4ccd-96b0-2893a7ec7136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:09.949763 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcbc08-dbbf-4ccd-96b0-2893a7ec7136 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:09.950035 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcbc08-dbbf-4ccd-96b0-2893a7ec7136 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:09.950311 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcbc08-dbbf-4ccd-96b0-2893a7ec7136 None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:09.951069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddcbc08-dbbf-4ccd-96b0-2893a7ec7136 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:09.951360 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddcbc08-dbbf-4ccd-96b0-2893a7ec7136 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:09.952799 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddcbc08-dbbf-4ccd-96b0-2893a7ec7136 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:09.960263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:09.966295 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 289/564] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:09.967532 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddcbc08-dbbf-4ccd-96b0-2893a7ec7136 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:10.000911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f1267b-c65a-49ae-8128-f9386f674007 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:10.039143 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec88e4-79c8-4928-be3e-d9af334a37bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:10.039916 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec88e4-79c8-4928-be3e-d9af334a37bc None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:10.040255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec88e4-79c8-4928-be3e-d9af334a37bc None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:10.040470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec88e4-79c8-4928-be3e-d9af334a37bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:10.050820 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec88e4-79c8-4928-be3e-d9af334a37bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:10.058435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-69ec88e4-79c8-4928-be3e-d9af334a37bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:10.058435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:10.058435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:10.058435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:10.058435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:10.058435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:10.058435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:10.058435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:10.058435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:10.058435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:10.058435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:10.058435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:10.058435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:10.058435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:10.058435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:10.058435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:10.059384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:10.059384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:10.059384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:10.059384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:10.059384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:10.059384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:10.059384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:10.059384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:10.059384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:10.059384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:10.059384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:10.059384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:10.059384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:10.059384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:10.059384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:10.059384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:10.060259 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:10.060259 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:10.060259 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:10.060259 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:10.060259 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:10.060259 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:10.060259 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:10.060259 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:10.060259 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:10.060259 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:10.060259 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:10.060259 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:10.060259 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 276/565] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:27:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:10.061025 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec88e4-79c8-4928-be3e-d9af334a37bc None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:10.079868 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:10.083716 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddcbc08-dbbf-4ccd-96b0-2893a7ec7136 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:10.119654 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dfdede-1d24-479f-81aa-f96fd51cad23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:10.120676 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfdede-1d24-479f-81aa-f96fd51cad23 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:10.121094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfdede-1d24-479f-81aa-f96fd51cad23 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:10.121378 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfdede-1d24-479f-81aa-f96fd51cad23 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:10.132319 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dfdede-1d24-479f-81aa-f96fd51cad23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:10.141562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-87dfdede-1d24-479f-81aa-f96fd51cad23 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:10.141562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:10.141562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:10.141562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:10.141562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:10.141562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:10.141562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:10.141562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:10.141562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:10.141562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:10.141562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:10.141562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:10.141562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:10.141562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:10.141562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:10.141562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:10.142544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:10.142544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:10.142544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:10.142544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:10.142544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:10.142544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:10.142544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:10.142544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:10.142544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:10.142544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:10.142544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:10.142544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:10.142544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:10.142544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:10.142544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:10.142544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:10.143363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:10.143363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:10.143363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:10.143363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:10.143363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:10.143363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:10.143363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:10.143363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:10.143363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:10.143363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:10.143363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:10.143363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:10.143363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 290/566] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:27:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:10.144659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dfdede-1d24-479f-81aa-f96fd51cad23 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:10.179609 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec88e4-79c8-4928-be3e-d9af334a37bc None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:10.208841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:10.218513 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33efe4-f927-4b77-bdae-5623cca23bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:10.219304 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33efe4-f927-4b77-bdae-5623cca23bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:10.219579 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33efe4-f927-4b77-bdae-5623cca23bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:10.223486 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33efe4-f927-4b77-bdae-5623cca23bc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:10.230890 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f33efe4-f927-4b77-bdae-5623cca23bc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:10.232665 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f33efe4-f927-4b77-bdae-5623cca23bc1 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:10.248681 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dfdede-1d24-479f-81aa-f96fd51cad23 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:10.250123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-6f33efe4-f927-4b77-bdae-5623cca23bc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:10.250123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:10.250123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:10.250123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:10.250123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:10.250123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:10.250123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:10.250123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:10.250123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:10.250123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:10.250123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:10.250123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:10.250123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:10.250123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:10.250123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:10.251203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:10.251203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:10.251203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:10.251203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:10.251203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:10.251203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:10.251203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:10.251203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:10.251203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:10.251203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:10.251203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:10.251203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:10.251203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:10.251203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:10.251203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:10.251203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:10.251203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:10.252303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:10.252303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:10.252303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:10.252303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:10.252303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 277/567] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:10.252303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33efe4-f927-4b77-bdae-5623cca23bc1 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:10.284651 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e8a4a-4437-4b66-a533-384510fa369a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:10.285438 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e8a4a-4437-4b66-a533-384510fa369a None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:10.285723 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e8a4a-4437-4b66-a533-384510fa369a None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:10.285998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e8a4a-4437-4b66-a533-384510fa369a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:10.292612 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e8a4a-4437-4b66-a533-384510fa369a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:10.294057 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e8a4a-4437-4b66-a533-384510fa369a None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:10.308539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-919e8a4a-4437-4b66-a533-384510fa369a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:10.308539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:10.308539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:10.308539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:10.308539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:10.308539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:10.308539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:10.308539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:10.308539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:10.308539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:10.308539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:10.308539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:10.308539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:10.308539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:10.308539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:10.309502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:10.309502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:10.309502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:10.309502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:10.309502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:10.309502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:10.309502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:10.309502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:10.309502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:10.309502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:10.309502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:10.309502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:10.309502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:10.309502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:10.309502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:10.309502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:10.309502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:10.310425 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:10.310425 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:10.310425 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:10.310425 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:10.310425 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:10.310425 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:10.310425 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:10.310425 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:10.310425 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 291/568] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:10.310923 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e8a4a-4437-4b66-a533-384510fa369a None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:10.329162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:10.359293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33efe4-f927-4b77-bdae-5623cca23bc1 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:10.394339 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6e454-79c5-4edc-8e82-200517aab152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:10.395026 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6e454-79c5-4edc-8e82-200517aab152 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:10.395305 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6e454-79c5-4edc-8e82-200517aab152 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:10.395582 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6e454-79c5-4edc-8e82-200517aab152 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:10.396316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6e454-79c5-4edc-8e82-200517aab152 None admin] RBAC: Adding query filter params (name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:10.396619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6e454-79c5-4edc-8e82-200517aab152 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:10.397975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6e454-79c5-4edc-8e82-200517aab152 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:10.405916 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 278/569] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Jun 27 18:27:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:10.407131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf4938-ff99-41e4-a9c0-302ffb5a9131 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:10.407415 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf4938-ff99-41e4-a9c0-302ffb5a9131 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:10.407691 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf4938-ff99-41e4-a9c0-302ffb5a9131 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:10.408898 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 279/570] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:10.409577 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf4938-ff99-41e4-a9c0-302ffb5a9131 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:10.420576 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e8a4a-4437-4b66-a533-384510fa369a None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:10.451540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:10.455789 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c60903-71ad-44fe-a16c-fb210a925b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:10.456462 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c60903-71ad-44fe-a16c-fb210a925b5f None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:10.456746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c60903-71ad-44fe-a16c-fb210a925b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:10.457020 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c60903-71ad-44fe-a16c-fb210a925b5f None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:10.457770 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c60903-71ad-44fe-a16c-fb210a925b5f None admin] RBAC: Adding query filter params (name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:10.458077 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c60903-71ad-44fe-a16c-fb210a925b5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:10.459488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c60903-71ad-44fe-a16c-fb210a925b5f None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:10.467770 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 292/571] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Jun 27 18:27:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:10.469009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c60903-71ad-44fe-a16c-fb210a925b5f None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:10.512505 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf4938-ff99-41e4-a9c0-302ffb5a9131 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:10.546501 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b0526-fda7-4fca-88a8-5bbe55539113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:10.547147 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b0526-fda7-4fca-88a8-5bbe55539113 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:10.547436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b0526-fda7-4fca-88a8-5bbe55539113 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:10.547707 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b0526-fda7-4fca-88a8-5bbe55539113 None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:10.548423 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b0526-fda7-4fca-88a8-5bbe55539113 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:10.548728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b0526-fda7-4fca-88a8-5bbe55539113 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:10.550093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b0526-fda7-4fca-88a8-5bbe55539113 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:10.553341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:10.562354 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 280/572] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:10.563265 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b0526-fda7-4fca-88a8-5bbe55539113 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:10.572476 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c60903-71ad-44fe-a16c-fb210a925b5f None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:10.606233 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1993ddc-f1f3-4cdc-bd8a-2c7cd5c8f2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:10.607168 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1993ddc-f1f3-4cdc-bd8a-2c7cd5c8f2ba None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:10.607521 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1993ddc-f1f3-4cdc-bd8a-2c7cd5c8f2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:10.607867 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1993ddc-f1f3-4cdc-bd8a-2c7cd5c8f2ba None admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:10.608735 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1993ddc-f1f3-4cdc-bd8a-2c7cd5c8f2ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:10.609040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1993ddc-f1f3-4cdc-bd8a-2c7cd5c8f2ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:10.610527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1993ddc-f1f3-4cdc-bd8a-2c7cd5c8f2ba None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:10.622909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 293/573] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:10.627691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c6085-9f91-4ab3-9399-2a65a9d870cf None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:10.627985 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c6085-9f91-4ab3-9399-2a65a9d870cf None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:10.628451 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c6085-9f91-4ab3-9399-2a65a9d870cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:10.643838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-ec2c6085-9f91-4ab3-9399-2a65a9d870cf None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:10.645024 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ec2c6085-9f91-4ab3-9399-2a65a9d870cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:10.658658 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:10.661765 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b0526-fda7-4fca-88a8-5bbe55539113 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:10.695405 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f5dec-8c62-4031-a5c2-501d96654ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:10.696186 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f5dec-8c62-4031-a5c2-501d96654ded None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:10.696468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f5dec-8c62-4031-a5c2-501d96654ded None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:10.696743 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f5dec-8c62-4031-a5c2-501d96654ded None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:10.707439 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f5dec-8c62-4031-a5c2-501d96654ded None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:10.708875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f5dec-8c62-4031-a5c2-501d96654ded None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:10.713460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 294/574] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:27:10.714760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c6085-9f91-4ab3-9399-2a65a9d870cf None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:10.726192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-fe4f5dec-8c62-4031-a5c2-501d96654ded None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:10.726192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:10.726192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:10.726192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:10.726192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:10.726192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:10.726192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:10.726192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:10.726192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:10.726192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:10.726192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:10.726192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:10.726192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:10.726192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:10.726192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:10.727093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:10.727093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:10.727093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:10.727093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:10.727093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:10.727093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:10.727093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:10.727093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:10.727093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:10.727093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:10.727093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:10.727093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:10.727093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:10.727093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:10.727093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:10.727093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:10.727093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:10.727972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:10.727972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:10.727972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:10.727972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:10.727972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 281/575] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:10.728249 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f5dec-8c62-4031-a5c2-501d96654ded None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:10.794330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:10.823614 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c6085-9f91-4ab3-9399-2a65a9d870cf None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:10.831430 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f5dec-8c62-4031-a5c2-501d96654ded None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:10.862027 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590508d-c229-4161-bc1f-33600f557523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:10.862771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590508d-c229-4161-bc1f-33600f557523 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:10.863057 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590508d-c229-4161-bc1f-33600f557523 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:10.863332 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590508d-c229-4161-bc1f-33600f557523 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:10.865535 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195ebdd-7491-401c-9eda-924bd3c637dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:10.866320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195ebdd-7491-401c-9eda-924bd3c637dc None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:10.866637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195ebdd-7491-401c-9eda-924bd3c637dc None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:10.866913 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195ebdd-7491-401c-9eda-924bd3c637dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:10.874174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590508d-c229-4161-bc1f-33600f557523 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:10.874291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195ebdd-7491-401c-9eda-924bd3c637dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:10.875656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195ebdd-7491-401c-9eda-924bd3c637dc None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:10.875755 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590508d-c229-4161-bc1f-33600f557523 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:10.889612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-6195ebdd-7491-401c-9eda-924bd3c637dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:10.889612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:10.889612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:10.889612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:10.889612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:10.889612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:10.889612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:10.889612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:10.889612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:10.889612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:10.889612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:10.889612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:10.889612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:10.889612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:10.889612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:10.890991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:10.890991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:10.890991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:10.890991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:10.890991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:10.890991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:10.890991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:10.890991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:10.890991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:10.890991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:10.890991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:10.890991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:10.890991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:10.890991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:10.890991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:10.890991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:10.890991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:10.891864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:10.891864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:10.891864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:10.891864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:10.891864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:10.891864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:10.891864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:10.891864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:10.891864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 282/576] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:10.892349 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195ebdd-7491-401c-9eda-924bd3c637dc None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:10.894290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-3590508d-c229-4161-bc1f-33600f557523 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:10.894290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:10.894290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:10.894290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:10.894290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:10.894290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:10.894290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:10.894290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:10.894290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:10.894290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:10.894290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:10.894290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:10.894290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:10.894290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:10.894290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:10.895132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:10.895132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:10.895132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:10.895132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:10.895132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:10.895132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:10.895132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:10.895132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:10.895132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:10.895132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:10.895132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:10.895132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:10.895132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:10.895132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:10.895132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:10.895132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:10.895132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:10.896003 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:10.896003 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:10.896003 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:10.896003 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:10.896003 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 295/577] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:10.896533 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590508d-c229-4161-bc1f-33600f557523 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:10.909065 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:10.993161 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590508d-c229-4161-bc1f-33600f557523 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:10.993681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195ebdd-7491-401c-9eda-924bd3c637dc None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:11.015384 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:11.025990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19736f74-5ba7-4fc4-a855-76b053ee6a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:11.027289 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19736f74-5ba7-4fc4-a855-76b053ee6a66 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:11.027425 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c0e7e-9eaf-4b7c-b8bd-11f6af6dbcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:11.027586 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19736f74-5ba7-4fc4-a855-76b053ee6a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:11.027862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19736f74-5ba7-4fc4-a855-76b053ee6a66 None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:11.028036 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c0e7e-9eaf-4b7c-b8bd-11f6af6dbcbc None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:11.028324 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c0e7e-9eaf-4b7c-b8bd-11f6af6dbcbc None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:11.028571 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19736f74-5ba7-4fc4-a855-76b053ee6a66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:11.028662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c0e7e-9eaf-4b7c-b8bd-11f6af6dbcbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:11.028872 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19736f74-5ba7-4fc4-a855-76b053ee6a66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:11.029442 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675c0e7e-9eaf-4b7c-b8bd-11f6af6dbcbc None admin] RBAC: Adding query filter params (role.id=c4bc6e72b9d647c5b2485d52d55f7dd4, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=a1529c9a45434a19968e924797688f06) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:11.029759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675c0e7e-9eaf-4b7c-b8bd-11f6af6dbcbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:11.030227 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19736f74-5ba7-4fc4-a855-76b053ee6a66 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:11.031208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675c0e7e-9eaf-4b7c-b8bd-11f6af6dbcbc None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:11.036642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 283/578] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:27:10 2022] GET /identity/v3/role_assignments?user.id=a1529c9a45434a19968e924797688f06&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=c4bc6e72b9d647c5b2485d52d55f7dd4 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:11.037408 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c0e7e-9eaf-4b7c-b8bd-11f6af6dbcbc None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:11.041648 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 296/579] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:11.042864 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19736f74-5ba7-4fc4-a855-76b053ee6a66 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:11.132264 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:11.159524 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c0e7e-9eaf-4b7c-b8bd-11f6af6dbcbc None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:11.165868 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19736f74-5ba7-4fc4-a855-76b053ee6a66 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:11.199750 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443af36d-ffab-490c-8639-ab87af32a0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:11.200751 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443af36d-ffab-490c-8639-ab87af32a0b5 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:11.201033 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443af36d-ffab-490c-8639-ab87af32a0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:11.201720 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443af36d-ffab-490c-8639-ab87af32a0b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:11.204112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fa4c7-b97e-4ae9-888a-9f66bd0e9124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:11.204677 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fa4c7-b97e-4ae9-888a-9f66bd0e9124 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:11.205409 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fa4c7-b97e-4ae9-888a-9f66bd0e9124 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:11.205409 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fa4c7-b97e-4ae9-888a-9f66bd0e9124 None admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:11.206428 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357fa4c7-b97e-4ae9-888a-9f66bd0e9124 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:11.206428 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357fa4c7-b97e-4ae9-888a-9f66bd0e9124 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:11.207858 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357fa4c7-b97e-4ae9-888a-9f66bd0e9124 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:11.214202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443af36d-ffab-490c-8639-ab87af32a0b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:11.220700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 297/580] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:11.225514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fa4c7-b97e-4ae9-888a-9f66bd0e9124 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:11.225640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-443af36d-ffab-490c-8639-ab87af32a0b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:11.225640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:11.225640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:11.225640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:11.225640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:11.225640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:11.225640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:11.225640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:11.225640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:11.225640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:11.225640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:11.225640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:11.225640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:11.225640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:11.225640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:11.225640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:11.226547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:11.226547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:11.226547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:11.226547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:11.226547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:11.226547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:11.226547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:11.226547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:11.226547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:11.226547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:11.226547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:11.226547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:11.226547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:11.226547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:11.226547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:11.226547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:11.227387 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:11.227387 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:11.227387 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:11.227387 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:11.227387 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:11.227387 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:11.227387 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:11.227387 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:11.227387 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:11.227387 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:11.227387 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:11.227387 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:11.227387 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 284/581] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:27:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:11.227387 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443af36d-ffab-490c-8639-ab87af32a0b5 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:11.272685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:11.376618 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fa4c7-b97e-4ae9-888a-9f66bd0e9124 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:11.381605 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443af36d-ffab-490c-8639-ab87af32a0b5 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:11.413814 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d96a0-803c-4855-8cc4-504da7ed7eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:11.414505 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d96a0-803c-4855-8cc4-504da7ed7eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:11.414790 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d96a0-803c-4855-8cc4-504da7ed7eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:11.415065 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d96a0-803c-4855-8cc4-504da7ed7eb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:11.415789 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d96a0-803c-4855-8cc4-504da7ed7eb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:11.416093 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d96a0-803c-4855-8cc4-504da7ed7eb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:11.419374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69c11c-42c0-495e-94ef-eefc0a53038c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:11.419765 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d96a0-803c-4855-8cc4-504da7ed7eb9 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:11.420502 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69c11c-42c0-495e-94ef-eefc0a53038c None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:11.421031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69c11c-42c0-495e-94ef-eefc0a53038c None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:11.421461 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69c11c-42c0-495e-94ef-eefc0a53038c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:11.424534 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:11.435493 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 298/582] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:11.435684 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac69c11c-42c0-495e-94ef-eefc0a53038c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:11.435778 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d96a0-803c-4855-8cc4-504da7ed7eb9 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:11.435904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac69c11c-42c0-495e-94ef-eefc0a53038c None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:11.451135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-ac69c11c-42c0-495e-94ef-eefc0a53038c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 18:27:11.451135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:11.451135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:11.451135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:11.451135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:11.451135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:11.451135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:11.451135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:11.451135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:11.451135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:11.451135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:11.451135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:11.451135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:11.451135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:11.451135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:11.451135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:11.452172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:11.452172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:11.452172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:11.452172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:11.452172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:11.452172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:11.452172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:11.452172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:11.452172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:11.452172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:11.452172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:11.452172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:11.452172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:11.452172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:11.452172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:11.452172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:11.452172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:11.452172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:11.453145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:11.453145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:11.453145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:11.453145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:11.453145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:11.453145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:11.453145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:11.453145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:11.453145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 18:27:11.453145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:11.453145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 285/583] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:27:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:11.453145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69c11c-42c0-495e-94ef-eefc0a53038c None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:11.555676 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d96a0-803c-4855-8cc4-504da7ed7eb9 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:11.557670 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:11.564459 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69c11c-42c0-495e-94ef-eefc0a53038c None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:11.591748 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ac49b-e45a-4bca-b698-d2ef47bfa987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:11.592890 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ac49b-e45a-4bca-b698-d2ef47bfa987 None admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:11.593181 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ac49b-e45a-4bca-b698-d2ef47bfa987 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:11.593461 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ac49b-e45a-4bca-b698-d2ef47bfa987 None admin] PATH_INFO: `/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/1dddd4efb44c4152bb4151f7715f2223/roles/c4bc6e72b9d647c5b2485d52d55f7dd4` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:11.602181 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc553b-b52c-4497-8d37-c6c4637aaa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:11.603196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc553b-b52c-4497-8d37-c6c4637aaa3f None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:11.603196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc553b-b52c-4497-8d37-c6c4637aaa3f None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:11.603396 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc553b-b52c-4497-8d37-c6c4637aaa3f None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:11.604131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc553b-b52c-4497-8d37-c6c4637aaa3f None admin] RBAC: Adding query filter params (name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:11.604443 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc553b-b52c-4497-8d37-c6c4637aaa3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:11.605803 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc553b-b52c-4497-8d37-c6c4637aaa3f None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:11.610470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ac49b-e45a-4bca-b698-d2ef47bfa987 None admin] RBAC: Authorizing `identity:create_grant(project_id=84ecf552f8364884b4aeff3428cb81e4, user_id=1dddd4efb44c4152bb4151f7715f2223, role_id=c4bc6e72b9d647c5b2485d52d55f7dd4)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:11.612375 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ac49b-e45a-4bca-b698-d2ef47bfa987 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:11.618629 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 286/584] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Jun 27 18:27:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:11.618629 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc553b-b52c-4497-8d37-c6c4637aaa3f None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:11.675514 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:11.698463 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 299/585] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:27:11 2022] PUT /identity/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/1dddd4efb44c4152bb4151f7715f2223/roles/c4bc6e72b9d647c5b2485d52d55f7dd4 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:27:11.698463 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ac49b-e45a-4bca-b698-d2ef47bfa987 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:11.751696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc553b-b52c-4497-8d37-c6c4637aaa3f None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:11.796235 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7911015-058a-4366-887d-986129aa30b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:11.797170 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7911015-058a-4366-887d-986129aa30b7 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:11.797462 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7911015-058a-4366-887d-986129aa30b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:11.797740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7911015-058a-4366-887d-986129aa30b7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:11.806888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7911015-058a-4366-887d-986129aa30b7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:11.809157 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7911015-058a-4366-887d-986129aa30b7 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:11.825044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-d7911015-058a-4366-887d-986129aa30b7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 27 18:27:11.825044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:11.825044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:11.825044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:11.825044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:11.825044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:11.825044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:11.825044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:11.825044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:11.825044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:11.825044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:11.825044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:11.825044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:11.825044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:11.825044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:11.825044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:11.826233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:11.826233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:11.826233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:11.826233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:11.826233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:11.826233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:11.826233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:11.826233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:11.826233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:11.826233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:11.826233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:11.826233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:11.826233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:11.826233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:11.826233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:11.826233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:11.826233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:11.826233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:11.827409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:11.827409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:11.827409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:11.827409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:11.827409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:11.827409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:11.827409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:11.827409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:11.827409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 27 18:27:11.827409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:11.830018 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 287/586] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:27:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:11.830018 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7911015-058a-4366-887d-986129aa30b7 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:11.846654 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ac49b-e45a-4bca-b698-d2ef47bfa987 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:11.889200 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bcff2d-2872-46d8-8e7b-a1eeebb315a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:11.890096 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcff2d-2872-46d8-8e7b-a1eeebb315a3 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:11.890541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcff2d-2872-46d8-8e7b-a1eeebb315a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:11.890866 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcff2d-2872-46d8-8e7b-a1eeebb315a3 None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:11.891838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bcff2d-2872-46d8-8e7b-a1eeebb315a3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:11.892213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bcff2d-2872-46d8-8e7b-a1eeebb315a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:11.893896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bcff2d-2872-46d8-8e7b-a1eeebb315a3 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:11.916479 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 300/587] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Jun 27 18:27:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:11.918148 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bcff2d-2872-46d8-8e7b-a1eeebb315a3 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:11.933906 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:11.951126 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7911015-058a-4366-887d-986129aa30b7 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:11.974376 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:11.997628 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd3231-0831-4d72-a98b-01fa378dae7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:11.998437 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd3231-0831-4d72-a98b-01fa378dae7c None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:11.998790 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd3231-0831-4d72-a98b-01fa378dae7c None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:11.998989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd3231-0831-4d72-a98b-01fa378dae7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:12.010263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd3231-0831-4d72-a98b-01fa378dae7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:12.011850 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd3231-0831-4d72-a98b-01fa378dae7c None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:12.030685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-aebd3231-0831-4d72-a98b-01fa378dae7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:12.030685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:12.030685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:12.030685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:12.030685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:12.030685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:12.030685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:12.030685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:12.030685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:12.030685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:12.030685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:12.030685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:12.030685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:12.030685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:12.030685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:12.031786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:12.031786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:12.031786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:12.031786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:12.031786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:12.031786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:12.031786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:12.031786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:12.031786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:12.031786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:12.031786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:12.031786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:12.031786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:12.031786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:12.031786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:12.031786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:12.031786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:12.032853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:12.032853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:12.032853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:12.032853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:12.032853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 288/588] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:12.033123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd3231-0831-4d72-a98b-01fa378dae7c None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:12.047913 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bcff2d-2872-46d8-8e7b-a1eeebb315a3 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:12.084890 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445f71a-82de-4499-8e1e-8d8cba0b510a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:12.085804 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445f71a-82de-4499-8e1e-8d8cba0b510a None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:12.086241 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445f71a-82de-4499-8e1e-8d8cba0b510a None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:12.086681 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445f71a-82de-4499-8e1e-8d8cba0b510a None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:12.087691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445f71a-82de-4499-8e1e-8d8cba0b510a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:12.088148 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445f71a-82de-4499-8e1e-8d8cba0b510a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:12.089955 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445f71a-82de-4499-8e1e-8d8cba0b510a None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:12.113278 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 301/589] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Jun 27 18:27:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:12.114445 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445f71a-82de-4499-8e1e-8d8cba0b510a None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:12.142069 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd3231-0831-4d72-a98b-01fa378dae7c None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:12.180258 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55495c4b-a13c-46cf-bed3-f16f3b60bb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:12.181101 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55495c4b-a13c-46cf-bed3-f16f3b60bb5b None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:12.181423 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55495c4b-a13c-46cf-bed3-f16f3b60bb5b None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:12.181659 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55495c4b-a13c-46cf-bed3-f16f3b60bb5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:12.188940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:12.193124 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55495c4b-a13c-46cf-bed3-f16f3b60bb5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:12.194696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55495c4b-a13c-46cf-bed3-f16f3b60bb5b None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:12.213497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-55495c4b-a13c-46cf-bed3-f16f3b60bb5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:12.213497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:12.213497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:12.213497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:12.213497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:12.213497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:12.213497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:12.213497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:12.213497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:12.213497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:12.213497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:12.213497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:12.213497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:12.213497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:12.213497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:12.215209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:12.215209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:12.215209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:12.215209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:12.215209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:12.215209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:12.215209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:12.215209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:12.215209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:12.215209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:12.215209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:12.215209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:12.215209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:12.215209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:12.215209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:12.215209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:12.215209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:12.216082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:12.216082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:12.216082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:12.216082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:12.220104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 289/590] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:12.221186 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55495c4b-a13c-46cf-bed3-f16f3b60bb5b None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:12.229112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445f71a-82de-4499-8e1e-8d8cba0b510a None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:12.249095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:12.267724 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b264826-6715-458c-a482-4df451b3ce1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:12.268679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b264826-6715-458c-a482-4df451b3ce1d None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:12.269141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b264826-6715-458c-a482-4df451b3ce1d None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:12.269594 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b264826-6715-458c-a482-4df451b3ce1d None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:12.270524 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b264826-6715-458c-a482-4df451b3ce1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:12.271008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b264826-6715-458c-a482-4df451b3ce1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:12.272566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b264826-6715-458c-a482-4df451b3ce1d None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:12.287141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 302/591] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:12.288322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b264826-6715-458c-a482-4df451b3ce1d None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:12.328576 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55495c4b-a13c-46cf-bed3-f16f3b60bb5b None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:12.380822 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b75f5-4579-4544-9a41-9532ede21669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:12.381665 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b75f5-4579-4544-9a41-9532ede21669 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:12.381950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b75f5-4579-4544-9a41-9532ede21669 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:12.382222 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b75f5-4579-4544-9a41-9532ede21669 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:12.394133 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b75f5-4579-4544-9a41-9532ede21669 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:12.395479 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b75f5-4579-4544-9a41-9532ede21669 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:12.413449 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-bc8b75f5-4579-4544-9a41-9532ede21669 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:12.413449 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:12.413449 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:12.413449 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:12.413449 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:12.413449 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:12.413449 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:12.413449 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:12.413449 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:12.413449 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:12.413449 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:12.413449 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:12.413449 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:12.413449 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:12.413449 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:12.414345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:12.414345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:12.414345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:12.414345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:12.414345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:12.414345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:12.414345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:12.414345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:12.414345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:12.414345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:12.414345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:12.414345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:12.414345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:12.414345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:12.414345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:12.414345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:12.414345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:12.415651 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:12.415651 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:12.415651 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:12.415651 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:12.420025 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 290/592] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:12.420883 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b75f5-4579-4544-9a41-9532ede21669 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:12.425403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b264826-6715-458c-a482-4df451b3ce1d None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:12.454605 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:12.463921 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03591ff2-3a7c-41cd-af10-2cd9c84e071e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:12.465500 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03591ff2-3a7c-41cd-af10-2cd9c84e071e None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:12.465952 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03591ff2-3a7c-41cd-af10-2cd9c84e071e None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:12.472077 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03591ff2-3a7c-41cd-af10-2cd9c84e071e None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:12.472077 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03591ff2-3a7c-41cd-af10-2cd9c84e071e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:12.472077 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03591ff2-3a7c-41cd-af10-2cd9c84e071e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:12.472077 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03591ff2-3a7c-41cd-af10-2cd9c84e071e None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:12.483327 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 303/593] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:12.484416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03591ff2-3a7c-41cd-af10-2cd9c84e071e None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:12.505136 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:12.532135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b75f5-4579-4544-9a41-9532ede21669 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:12.568489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12efe414-c4dd-4b25-91a1-218c1b66c968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:12.569441 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efe414-c4dd-4b25-91a1-218c1b66c968 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:12.570159 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efe414-c4dd-4b25-91a1-218c1b66c968 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:12.570159 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12efe414-c4dd-4b25-91a1-218c1b66c968 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:12.578641 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12efe414-c4dd-4b25-91a1-218c1b66c968 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:12.580270 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12efe414-c4dd-4b25-91a1-218c1b66c968 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:12.595140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-12efe414-c4dd-4b25-91a1-218c1b66c968 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 18:27:12.595140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:12.595140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:12.595140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:12.595140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:12.595140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:12.595140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:12.595140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:12.595140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:12.595140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:12.595140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:12.595140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:12.595140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:12.595140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:12.595140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:12.595140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:12.596164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:12.596164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:12.596164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:12.596164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:12.596164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:12.596164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:12.596164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:12.596164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:12.596164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:12.596164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:12.596164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:12.596164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:12.596164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:12.596164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:12.596164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:12.596164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:12.596164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:12.596164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:12.597089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:12.597089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:12.597089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:12.597089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:12.597089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:12.597089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:12.597089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:12.597089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:12.597089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 18:27:12.597089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:12.597089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 291/594] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:27:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:12.597663 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12efe414-c4dd-4b25-91a1-218c1b66c968 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:12.603279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03591ff2-3a7c-41cd-af10-2cd9c84e071e None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:12.646962 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ffda5-5316-4f82-9ab1-295ecd1020dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:12.647747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ffda5-5316-4f82-9ab1-295ecd1020dc None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:12.648130 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ffda5-5316-4f82-9ab1-295ecd1020dc None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:12.648476 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ffda5-5316-4f82-9ab1-295ecd1020dc None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:12.649270 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ffda5-5316-4f82-9ab1-295ecd1020dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:12.649645 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ffda5-5316-4f82-9ab1-295ecd1020dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:12.652430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ffda5-5316-4f82-9ab1-295ecd1020dc None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:12.670535 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 304/595] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:12.671638 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ffda5-5316-4f82-9ab1-295ecd1020dc None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:12.732081 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12efe414-c4dd-4b25-91a1-218c1b66c968 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:12.733953 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:12.776993 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37773def-09ab-4f33-8cde-b3ff337ece25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:12.778144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37773def-09ab-4f33-8cde-b3ff337ece25 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:12.778652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37773def-09ab-4f33-8cde-b3ff337ece25 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:12.778940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37773def-09ab-4f33-8cde-b3ff337ece25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:12.781527 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:12.787101 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37773def-09ab-4f33-8cde-b3ff337ece25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:12.788725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37773def-09ab-4f33-8cde-b3ff337ece25 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:12.800924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ffda5-5316-4f82-9ab1-295ecd1020dc None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:12.805135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-37773def-09ab-4f33-8cde-b3ff337ece25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:12.805135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:12.805135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:12.805135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:12.805135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:12.805135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:12.805135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:12.805135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:12.805135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:12.805135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:12.805135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:12.805135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:12.805135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:12.805135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:12.805135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:12.806003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:12.806003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:12.806003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:12.806003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:12.806003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:12.806003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:12.806003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:12.806003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:12.806003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:12.806003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:12.806003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:12.806003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:12.806003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:12.806003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:12.806003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:12.806003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:12.806003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:12.806908 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:12.806908 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:12.806908 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:12.806908 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:12.806908 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:12.806908 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:12.806908 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:12.806908 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:12.811152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 292/596] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:12.812006 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37773def-09ab-4f33-8cde-b3ff337ece25 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:12.842982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe84fd-db09-44eb-b8bf-5e8ebf267ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:12.844121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe84fd-db09-44eb-b8bf-5e8ebf267ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:12.844608 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe84fd-db09-44eb-b8bf-5e8ebf267ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:12.845093 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe84fd-db09-44eb-b8bf-5e8ebf267ec6 None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:12.846239 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe84fd-db09-44eb-b8bf-5e8ebf267ec6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:12.846921 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe84fd-db09-44eb-b8bf-5e8ebf267ec6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:12.848571 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe84fd-db09-44eb-b8bf-5e8ebf267ec6 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:12.868602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 305/597] 10.222.0.15 () {56 vars in 1139 bytes} [Mon Jun 27 18:27:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:12.869814 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe84fd-db09-44eb-b8bf-5e8ebf267ec6 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:12.924952 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37773def-09ab-4f33-8cde-b3ff337ece25 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:12.962613 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226a88d-c01a-4ca5-9bd7-7c2eccc0fd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:12.963562 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226a88d-c01a-4ca5-9bd7-7c2eccc0fd9f None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:12.963851 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226a88d-c01a-4ca5-9bd7-7c2eccc0fd9f None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:12.964684 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226a88d-c01a-4ca5-9bd7-7c2eccc0fd9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:12.972283 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8226a88d-c01a-4ca5-9bd7-7c2eccc0fd9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:12.973728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8226a88d-c01a-4ca5-9bd7-7c2eccc0fd9f None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:12.981317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe84fd-db09-44eb-b8bf-5e8ebf267ec6 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:12.989528 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:12.990228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-8226a88d-c01a-4ca5-9bd7-7c2eccc0fd9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:12.990228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:12.990228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:12.990228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:12.990228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:12.990228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:12.990228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:12.990228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:12.990228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:12.990228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:12.990228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:12.990228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:12.990228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:12.990228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:12.990228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:12.991198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:12.991198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:12.991198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:12.991198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:12.991198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:12.991198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:12.991198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:12.991198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:12.991198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:12.991198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:12.991198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:12.991198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:12.991198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:12.991198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:12.991198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:12.991198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:12.991198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:12.992082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:12.992082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:12.992082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:12.992082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:12.992082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:12.992082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:12.992082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:12.992082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:12.996016 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 293/598] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:12.996863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226a88d-c01a-4ca5-9bd7-7c2eccc0fd9f None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:13.022937 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299517e2-0811-49b3-b0cb-ada37b9cd433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:13.024065 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299517e2-0811-49b3-b0cb-ada37b9cd433 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:13.024552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299517e2-0811-49b3-b0cb-ada37b9cd433 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:13.025072 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299517e2-0811-49b3-b0cb-ada37b9cd433 None admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:13.026058 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299517e2-0811-49b3-b0cb-ada37b9cd433 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:13.026813 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299517e2-0811-49b3-b0cb-ada37b9cd433 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:13.028763 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299517e2-0811-49b3-b0cb-ada37b9cd433 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:13.041301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 306/599] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:13.043031 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299517e2-0811-49b3-b0cb-ada37b9cd433 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:13.063740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:13.122840 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226a88d-c01a-4ca5-9bd7-7c2eccc0fd9f None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:13.161261 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713f762-5841-4001-9309-6932037b8754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:13.162045 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713f762-5841-4001-9309-6932037b8754 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:13.162219 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299517e2-0811-49b3-b0cb-ada37b9cd433 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:13.162684 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713f762-5841-4001-9309-6932037b8754 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:13.162684 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713f762-5841-4001-9309-6932037b8754 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:13.175653 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713f762-5841-4001-9309-6932037b8754 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:13.177122 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713f762-5841-4001-9309-6932037b8754 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:13.195161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-b713f762-5841-4001-9309-6932037b8754 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:13.195161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:13.195161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:13.195161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:13.195161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:13.195161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:13.195161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:13.195161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:13.195161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:13.195161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:13.195161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:13.195161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:13.195161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:13.195161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:13.195161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:13.196063 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:13.196063 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:13.196063 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:13.196063 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:13.196063 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:13.196063 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:13.196063 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:13.196063 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:13.196063 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:13.196063 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:13.196063 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:13.196063 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:13.196063 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:13.196063 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:13.196063 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:13.196063 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:13.196063 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:13.196975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:13.196975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:13.196975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:13.196975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:13.198589 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 294/600] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:13.199467 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713f762-5841-4001-9309-6932037b8754 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:13.204574 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46f11b-3d10-4fea-a8bd-714d230ae298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:13.205239 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46f11b-3d10-4fea-a8bd-714d230ae298 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:13.205519 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46f11b-3d10-4fea-a8bd-714d230ae298 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:13.205791 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46f11b-3d10-4fea-a8bd-714d230ae298 None admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:13.206602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46f11b-3d10-4fea-a8bd-714d230ae298 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:13.206910 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46f11b-3d10-4fea-a8bd-714d230ae298 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:13.208330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46f11b-3d10-4fea-a8bd-714d230ae298 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:13.226275 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 307/601] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:13.226275 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a69687-f341-4d18-95b0-15c4842a36bf None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:13.226275 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a69687-f341-4d18-95b0-15c4842a36bf None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:13.226275 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a69687-f341-4d18-95b0-15c4842a36bf None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:13.226275 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 308/602] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:13.230277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a69687-f341-4d18-95b0-15c4842a36bf None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:13.257220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:13.315221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713f762-5841-4001-9309-6932037b8754 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:13.322687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:13.337353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a69687-f341-4d18-95b0-15c4842a36bf None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:13.351792 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd730b9-54b6-438f-9898-f1da2ebb7e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:13.352671 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd730b9-54b6-438f-9898-f1da2ebb7e4e None admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:13.352953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd730b9-54b6-438f-9898-f1da2ebb7e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:13.353230 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd730b9-54b6-438f-9898-f1da2ebb7e4e None admin] PATH_INFO: `/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/c4f6f8e0a0cb4c9a9dd0d99fc0022235/roles/c4bc6e72b9d647c5b2485d52d55f7dd4` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:13.368106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd730b9-54b6-438f-9898-f1da2ebb7e4e None admin] RBAC: Authorizing `identity:create_grant(project_id=84ecf552f8364884b4aeff3428cb81e4, user_id=c4f6f8e0a0cb4c9a9dd0d99fc0022235, role_id=c4bc6e72b9d647c5b2485d52d55f7dd4)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:13.369539 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd730b9-54b6-438f-9898-f1da2ebb7e4e None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:13.375857 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6b1cd-38b5-48a0-b322-e9fcb0ba7e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:13.376515 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6b1cd-38b5-48a0-b322-e9fcb0ba7e6e None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:13.376799 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6b1cd-38b5-48a0-b322-e9fcb0ba7e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:13.377104 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6b1cd-38b5-48a0-b322-e9fcb0ba7e6e None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:13.377806 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6b1cd-38b5-48a0-b322-e9fcb0ba7e6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:13.378112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6b1cd-38b5-48a0-b322-e9fcb0ba7e6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:13.379520 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6b1cd-38b5-48a0-b322-e9fcb0ba7e6e None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:13.390756 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 309/603] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:13.391533 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6b1cd-38b5-48a0-b322-e9fcb0ba7e6e None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:13.438588 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 295/604] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:27:13 2022] PUT /identity/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/c4f6f8e0a0cb4c9a9dd0d99fc0022235/roles/c4bc6e72b9d647c5b2485d52d55f7dd4 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:27:13.438588 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53830e-00d5-429e-8321-9c7fb5f526b3 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:13.438588 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53830e-00d5-429e-8321-9c7fb5f526b3 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:13.438588 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53830e-00d5-429e-8321-9c7fb5f526b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:13.464466 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-7b53830e-00d5-429e-8321-9c7fb5f526b3 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:13.466003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-7b53830e-00d5-429e-8321-9c7fb5f526b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:13.495764 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:13.515105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6b1cd-38b5-48a0-b322-e9fcb0ba7e6e None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:13.555415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0eefa3-3cba-4b77-8b80-c06ed8bcb4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:13.556679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0eefa3-3cba-4b77-8b80-c06ed8bcb4ca None admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:13.557145 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0eefa3-3cba-4b77-8b80-c06ed8bcb4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:13.557564 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0eefa3-3cba-4b77-8b80-c06ed8bcb4ca None admin] PATH_INFO: `/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/75e75e5ae3ca45d0b503f6e52f399558/roles/c4bc6e72b9d647c5b2485d52d55f7dd4` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:13.574416 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 296/605] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:27:13.574416 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53830e-00d5-429e-8321-9c7fb5f526b3 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:13.587283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0eefa3-3cba-4b77-8b80-c06ed8bcb4ca None admin] RBAC: Authorizing `identity:create_grant(project_id=84ecf552f8364884b4aeff3428cb81e4, user_id=75e75e5ae3ca45d0b503f6e52f399558, role_id=c4bc6e72b9d647c5b2485d52d55f7dd4)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:13.589734 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0eefa3-3cba-4b77-8b80-c06ed8bcb4ca None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:13.677707 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:13.679116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 310/606] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:27:13 2022] PUT /identity/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/75e75e5ae3ca45d0b503f6e52f399558/roles/c4bc6e72b9d647c5b2485d52d55f7dd4 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:27:13.680027 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0eefa3-3cba-4b77-8b80-c06ed8bcb4ca None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:13.712862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53830e-00d5-429e-8321-9c7fb5f526b3 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:13.801182 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01747116-88ee-4688-b99a-d4254926a64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:13.803657 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01747116-88ee-4688-b99a-d4254926a64a None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:13.805580 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01747116-88ee-4688-b99a-d4254926a64a None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:13.805940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01747116-88ee-4688-b99a-d4254926a64a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:13.823195 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01747116-88ee-4688-b99a-d4254926a64a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:13.825417 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01747116-88ee-4688-b99a-d4254926a64a None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:13.857342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-01747116-88ee-4688-b99a-d4254926a64a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:13.857342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:13.857342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:13.857342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:13.857342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:13.857342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:13.857342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:13.857342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:13.857342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:13.857342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:13.857342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:13.857342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:13.857342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:13.857342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:13.857342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:13.858297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:13.858297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:13.858297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:13.858297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:13.858297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:13.858297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:13.858297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:13.858297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:13.858297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:13.858297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:13.858297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:13.858297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:13.858297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:13.858297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:13.858297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:13.858297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:13.858297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:13.859224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:13.859224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:13.859224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:13.859224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:13.859224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:13.859224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:13.859224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:13.859224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:13.861014 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 297/607] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:13.866046 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01747116-88ee-4688-b99a-d4254926a64a None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:13.895223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0eefa3-3cba-4b77-8b80-c06ed8bcb4ca None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:13.941477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dcf27a-f4df-46ec-9b07-3e38202558e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:13.942313 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcf27a-f4df-46ec-9b07-3e38202558e6 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:13.943163 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcf27a-f4df-46ec-9b07-3e38202558e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:13.943441 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcf27a-f4df-46ec-9b07-3e38202558e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:13.946784 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:13.958657 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dcf27a-f4df-46ec-9b07-3e38202558e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:13.970615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-95dcf27a-f4df-46ec-9b07-3e38202558e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:13.970615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:13.970615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:13.970615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:13.970615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:13.970615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:13.970615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:13.970615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:13.970615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:13.970615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:13.970615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:13.970615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:13.970615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:13.970615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:13.970615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:13.970615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:13.971582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:13.971582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:13.971582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:13.971582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:13.971582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:13.971582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:13.971582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:13.971582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:13.971582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:13.971582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:13.971582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:13.971582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:13.971582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:13.971582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:13.971582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:13.971582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:13.972420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:13.972420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:13.972420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:13.972420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:13.972420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:13.972420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:13.972420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:13.972420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:13.972420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:13.972420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:13.972420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:13.972420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:13.975278 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 311/608] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:27:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:13.977169 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d6caa-da34-46e9-b399-a9c24dec4823 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:13.983980 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d6caa-da34-46e9-b399-a9c24dec4823 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:13.983980 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d6caa-da34-46e9-b399-a9c24dec4823 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:13.985079 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 312/609] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:13 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:13.988242 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01747116-88ee-4688-b99a-d4254926a64a None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:13.989643 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d6caa-da34-46e9-b399-a9c24dec4823 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:14.025069 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de571e13-1184-44db-bc95-a67e1ac02bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:14.025721 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de571e13-1184-44db-bc95-a67e1ac02bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:14.026001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de571e13-1184-44db-bc95-a67e1ac02bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:14.026276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de571e13-1184-44db-bc95-a67e1ac02bb3 None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:14.027663 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de571e13-1184-44db-bc95-a67e1ac02bb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:14.027968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de571e13-1184-44db-bc95-a67e1ac02bb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:14.029411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de571e13-1184-44db-bc95-a67e1ac02bb3 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:14.046682 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 298/610] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:14.048060 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fea57-2bae-4e28-826e-9e9c08ed625a None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:14.048346 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fea57-2bae-4e28-826e-9e9c08ed625a None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:14.048622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fea57-2bae-4e28-826e-9e9c08ed625a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:14.061232 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:14.067814 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-be1fea57-2bae-4e28-826e-9e9c08ed625a None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:14.069112 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-be1fea57-2bae-4e28-826e-9e9c08ed625a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:14.112610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d6caa-da34-46e9-b399-a9c24dec4823 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:14.150153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213b888-d212-48b1-aadd-0cc3c7d6f10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:14.151222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213b888-d212-48b1-aadd-0cc3c7d6f10b None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:14.152255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213b888-d212-48b1-aadd-0cc3c7d6f10b None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:14.152739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213b888-d212-48b1-aadd-0cc3c7d6f10b None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:14.153784 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213b888-d212-48b1-aadd-0cc3c7d6f10b None admin] RBAC: Adding query filter params (name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:14.154271 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213b888-d212-48b1-aadd-0cc3c7d6f10b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:14.155852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213b888-d212-48b1-aadd-0cc3c7d6f10b None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:14.158515 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 299/611] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:27:14.159540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1fea57-2bae-4e28-826e-9e9c08ed625a None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:14.174650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 313/612] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Jun 27 18:27:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:14.178541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213b888-d212-48b1-aadd-0cc3c7d6f10b None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:14.251951 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:14.295357 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213b888-d212-48b1-aadd-0cc3c7d6f10b None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:14.296261 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1fea57-2bae-4e28-826e-9e9c08ed625a None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:14.331455 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbacc59-8cfb-44dd-a7e2-b00727416317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:14.332509 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbacc59-8cfb-44dd-a7e2-b00727416317 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:14.332972 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbacc59-8cfb-44dd-a7e2-b00727416317 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:14.333314 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c43fd-6854-4691-8be3-b23a0cd3e6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:14.333635 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbacc59-8cfb-44dd-a7e2-b00727416317 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:14.334373 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c43fd-6854-4691-8be3-b23a0cd3e6e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:14.334766 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c43fd-6854-4691-8be3-b23a0cd3e6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:14.335431 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c43fd-6854-4691-8be3-b23a0cd3e6e1 None admin] PATH_INFO: `/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/dbe621916dbe4f1a8686097fe2bdf762/roles/c4bc6e72b9d647c5b2485d52d55f7dd4` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:14.339271 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:14.346711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbacc59-8cfb-44dd-a7e2-b00727416317 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:14.348387 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbacc59-8cfb-44dd-a7e2-b00727416317 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:14.351560 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c43fd-6854-4691-8be3-b23a0cd3e6e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=84ecf552f8364884b4aeff3428cb81e4, user_id=dbe621916dbe4f1a8686097fe2bdf762, role_id=c4bc6e72b9d647c5b2485d52d55f7dd4)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:14.353230 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c43fd-6854-4691-8be3-b23a0cd3e6e1 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:14.368220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-0bbacc59-8cfb-44dd-a7e2-b00727416317 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:14.368220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:14.368220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:14.368220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:14.368220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:14.368220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:14.368220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:14.368220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:14.368220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:14.368220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:14.368220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:14.368220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:14.368220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:14.368220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:14.368220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:14.369149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:14.369149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:14.369149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:14.369149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:14.369149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:14.369149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:14.369149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:14.369149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:14.369149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:14.369149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:14.369149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:14.369149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:14.369149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:14.369149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:14.369149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:14.369149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:14.369149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:14.370023 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:14.370023 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:14.370023 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:14.370023 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:14.374343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 314/613] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:14.375465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbacc59-8cfb-44dd-a7e2-b00727416317 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:14.408070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 300/614] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:27:14 2022] PUT /identity/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/dbe621916dbe4f1a8686097fe2bdf762/roles/c4bc6e72b9d647c5b2485d52d55f7dd4 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:27:14.409672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c43fd-6854-4691-8be3-b23a0cd3e6e1 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:14.538605 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c43fd-6854-4691-8be3-b23a0cd3e6e1 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:14.574906 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbacc59-8cfb-44dd-a7e2-b00727416317 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:14.583573 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea496a-b609-44db-a9ea-c884be455aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:14.585248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea496a-b609-44db-a9ea-c884be455aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:14.585806 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea496a-b609-44db-a9ea-c884be455aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:14.586307 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea496a-b609-44db-a9ea-c884be455aa4 None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:14.587286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea496a-b609-44db-a9ea-c884be455aa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:14.587796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea496a-b609-44db-a9ea-c884be455aa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:14.589493 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea496a-b609-44db-a9ea-c884be455aa4 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:14.600501 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:14.606246 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 301/615] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:14.612547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea496a-b609-44db-a9ea-c884be455aa4 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:14.669623 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:14.684608 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53496a89-b8f6-4a91-bdae-f6de5b4fbbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:14.685873 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53496a89-b8f6-4a91-bdae-f6de5b4fbbd8 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:14.686366 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53496a89-b8f6-4a91-bdae-f6de5b4fbbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:14.686932 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53496a89-b8f6-4a91-bdae-f6de5b4fbbd8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:14.697976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53496a89-b8f6-4a91-bdae-f6de5b4fbbd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:14.707875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-53496a89-b8f6-4a91-bdae-f6de5b4fbbd8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:14.707875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:14.707875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:14.707875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:14.707875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:14.707875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:14.707875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:14.707875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:14.707875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:14.707875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:14.707875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:14.707875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:14.707875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:14.707875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:14.707875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:14.707875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:14.708847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:14.708847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:14.708847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:14.708847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:14.708847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:14.708847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:14.708847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:14.708847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:14.708847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:14.708847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:14.708847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:14.708847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:14.708847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:14.708847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:14.708847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:14.708847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:14.709693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:14.709693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:14.709693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:14.709693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:14.709693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:14.709693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:14.709693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:14.709693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:14.709693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:14.709693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:14.709693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:14.709693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:14.712504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 315/616] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:27:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:14.718829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53496a89-b8f6-4a91-bdae-f6de5b4fbbd8 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:14.730566 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea496a-b609-44db-a9ea-c884be455aa4 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:14.772480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c7ae3-4f64-408b-9fa4-cf855447c8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:14.773338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c7ae3-4f64-408b-9fa4-cf855447c8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:14.773875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c7ae3-4f64-408b-9fa4-cf855447c8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:14.776932 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c7ae3-4f64-408b-9fa4-cf855447c8c6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:14.783612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c7ae3-4f64-408b-9fa4-cf855447c8c6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:14.785698 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c7ae3-4f64-408b-9fa4-cf855447c8c6 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:14.802205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-790c7ae3-4f64-408b-9fa4-cf855447c8c6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 18:27:14.802205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:14.802205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:14.802205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:14.802205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:14.802205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:14.802205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:14.802205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:14.802205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:14.802205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:14.802205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:14.802205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:14.802205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:14.802205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:14.802205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:14.802205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:14.803389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:14.803389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:14.803389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:14.803389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:14.803389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:14.803389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:14.803389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:14.803389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:14.803389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:14.803389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:14.803389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:14.803389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:14.803389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:14.803389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:14.803389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:14.803389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:14.803389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:14.803389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:14.804466 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:14.804466 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:14.804466 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:14.804466 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:14.804466 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:14.804466 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:14.804466 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:14.804466 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:14.804466 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 18:27:14.804466 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:14.805766 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 302/617] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:27:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:14.811754 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c7ae3-4f64-408b-9fa4-cf855447c8c6 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:14.912466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53496a89-b8f6-4a91-bdae-f6de5b4fbbd8 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:14.930468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c7ae3-4f64-408b-9fa4-cf855447c8c6 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:14.939282 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:14.969964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25117702-5d0b-431c-a83e-d0b5d63413b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:14.970820 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25117702-5d0b-431c-a83e-d0b5d63413b7 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:14.971557 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25117702-5d0b-431c-a83e-d0b5d63413b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:14.971940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25117702-5d0b-431c-a83e-d0b5d63413b7 None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:14.973168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25117702-5d0b-431c-a83e-d0b5d63413b7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:14.973577 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25117702-5d0b-431c-a83e-d0b5d63413b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:14.975287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25117702-5d0b-431c-a83e-d0b5d63413b7 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:14.986566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ff5a2-493e-4036-9b66-81448219de89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:14.988441 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ff5a2-493e-4036-9b66-81448219de89 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:14.988977 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ff5a2-493e-4036-9b66-81448219de89 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:14.989392 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ff5a2-493e-4036-9b66-81448219de89 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:14.990290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ff5a2-493e-4036-9b66-81448219de89 None admin] RBAC: Adding query filter params (name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:14.990755 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ff5a2-493e-4036-9b66-81448219de89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:14.992290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ff5a2-493e-4036-9b66-81448219de89 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:14.997516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 303/618] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Jun 27 18:27:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:15.003474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25117702-5d0b-431c-a83e-d0b5d63413b7 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:15.005532 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 316/619] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Jun 27 18:27:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:15.011936 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ff5a2-493e-4036-9b66-81448219de89 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:15.055570 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:15.161273 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ff5a2-493e-4036-9b66-81448219de89 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:15.212398 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25117702-5d0b-431c-a83e-d0b5d63413b7 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:15.233738 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67c7c0-c6ad-41ee-a3a1-5d8cfb519d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:15.235773 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67c7c0-c6ad-41ee-a3a1-5d8cfb519d28 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:15.236056 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67c7c0-c6ad-41ee-a3a1-5d8cfb519d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:15.236335 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67c7c0-c6ad-41ee-a3a1-5d8cfb519d28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:15.248794 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d67c7c0-c6ad-41ee-a3a1-5d8cfb519d28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:15.250267 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d67c7c0-c6ad-41ee-a3a1-5d8cfb519d28 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:15.268052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-6d67c7c0-c6ad-41ee-a3a1-5d8cfb519d28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:15.268052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:15.268052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:15.268052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:15.268052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:15.268052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:15.268052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:15.268052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:15.268052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:15.268052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:15.268052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:15.268052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:15.268052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:15.268052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:15.268052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:15.269069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:15.269069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:15.269069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:15.269069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:15.269069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:15.269069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:15.269069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:15.269069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:15.269069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:15.269069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:15.269069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:15.269069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:15.269069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:15.269069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:15.269069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:15.269069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:15.269069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:15.269944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:15.269944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:15.269944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:15.269944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:15.269944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 317/620] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:15.272035 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d16d10-8ebb-44d4-b5bd-4954f9c1ae0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:15.272833 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d16d10-8ebb-44d4-b5bd-4954f9c1ae0f None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:15.273120 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d16d10-8ebb-44d4-b5bd-4954f9c1ae0f None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:15.273397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d16d10-8ebb-44d4-b5bd-4954f9c1ae0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:15.276891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67c7c0-c6ad-41ee-a3a1-5d8cfb519d28 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:15.297115 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d16d10-8ebb-44d4-b5bd-4954f9c1ae0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:15.298986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d16d10-8ebb-44d4-b5bd-4954f9c1ae0f None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:15.329185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-34d16d10-8ebb-44d4-b5bd-4954f9c1ae0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:15.329185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:15.329185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:15.329185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:15.329185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:15.329185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:15.329185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:15.329185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:15.329185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:15.329185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:15.329185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:15.329185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:15.329185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:15.329185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:15.329185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:15.330144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:15.330144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:15.330144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:15.330144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:15.330144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:15.330144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:15.330144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:15.330144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:15.330144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:15.330144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:15.330144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:15.330144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:15.330144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:15.330144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:15.330144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:15.330144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:15.330144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:15.331059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:15.331059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:15.331059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:15.331059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:15.341942 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 304/621] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:15.349754 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d16d10-8ebb-44d4-b5bd-4954f9c1ae0f None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:15.377828 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:15.449412 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:15.475415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67c7c0-c6ad-41ee-a3a1-5d8cfb519d28 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:15.528045 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59c89e-6595-4d57-999f-be25f746d2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:15.528711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59c89e-6595-4d57-999f-be25f746d2dd None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:15.528996 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59c89e-6595-4d57-999f-be25f746d2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:15.529272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59c89e-6595-4d57-999f-be25f746d2dd None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:15.530001 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59c89e-6595-4d57-999f-be25f746d2dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:15.530309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59c89e-6595-4d57-999f-be25f746d2dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:15.531707 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59c89e-6595-4d57-999f-be25f746d2dd None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:15.535765 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d16d10-8ebb-44d4-b5bd-4954f9c1ae0f None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:15.558560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 318/622] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:15.563887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59c89e-6595-4d57-999f-be25f746d2dd None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:15.599886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1b845-6022-445f-b107-32cfe1e1f1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:15.600545 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1b845-6022-445f-b107-32cfe1e1f1fa None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:15.600827 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1b845-6022-445f-b107-32cfe1e1f1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:15.601105 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1b845-6022-445f-b107-32cfe1e1f1fa None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:15.601836 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1b845-6022-445f-b107-32cfe1e1f1fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:15.602143 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1b845-6022-445f-b107-32cfe1e1f1fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:15.603695 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1b845-6022-445f-b107-32cfe1e1f1fa None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:15.616576 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 305/623] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:15.622894 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1b845-6022-445f-b107-32cfe1e1f1fa None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:15.753725 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59c89e-6595-4d57-999f-be25f746d2dd None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:15.784120 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:15.803244 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea390aca-6101-49b4-8e01-a49c7a4c6465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:15.804617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea390aca-6101-49b4-8e01-a49c7a4c6465 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:15.805212 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea390aca-6101-49b4-8e01-a49c7a4c6465 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:15.805966 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea390aca-6101-49b4-8e01-a49c7a4c6465 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:15.808639 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1b845-6022-445f-b107-32cfe1e1f1fa None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:15.815682 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea390aca-6101-49b4-8e01-a49c7a4c6465 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:15.817752 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea390aca-6101-49b4-8e01-a49c7a4c6465 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:15.837506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-ea390aca-6101-49b4-8e01-a49c7a4c6465 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 18:27:15.837506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:15.837506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:15.837506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:15.837506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:15.837506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:15.837506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:15.837506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:15.837506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:15.837506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:15.837506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:15.837506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:15.837506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:15.837506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:15.837506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:15.837506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:15.838705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:15.838705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:15.838705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:15.838705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:15.838705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:15.838705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:15.838705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:15.838705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:15.838705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:15.838705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:15.838705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:15.838705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:15.838705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:15.838705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:15.838705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:15.838705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:15.838705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:15.838705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:15.840537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:15.840537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:15.840537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:15.840537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:15.840537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:15.840537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:15.840537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:15.840537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:15.840537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 18:27:15.840537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:15.841094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 319/624] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Jun 27 18:27:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:27:15.846444 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500f33a-4472-46ec-b4ce-e4173107869a None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:15.846444 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500f33a-4472-46ec-b4ce-e4173107869a None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:15.846444 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500f33a-4472-46ec-b4ce-e4173107869a None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:15.848131 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 320/625] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:15.851589 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500f33a-4472-46ec-b4ce-e4173107869a None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:15.856955 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:15.858190 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e87769-98d6-43da-8b76-621890ac50c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:15.859240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e87769-98d6-43da-8b76-621890ac50c8 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:15.859748 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e87769-98d6-43da-8b76-621890ac50c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:15.860246 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e87769-98d6-43da-8b76-621890ac50c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:15.868068 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e87769-98d6-43da-8b76-621890ac50c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:15.869761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e87769-98d6-43da-8b76-621890ac50c8 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:15.885446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-52e87769-98d6-43da-8b76-621890ac50c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:15.885446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:15.885446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:15.885446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:15.885446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:15.885446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:15.885446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:15.885446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:15.885446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:15.885446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:15.885446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:15.885446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:15.885446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:15.885446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:15.885446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:15.887102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:15.887102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:15.887102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:15.887102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:15.887102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:15.887102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:15.887102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:15.887102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:15.887102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:15.887102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:15.887102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:15.887102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:15.887102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:15.887102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:15.887102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:15.887102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:15.887102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:15.888169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:15.888169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:15.888169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:15.888169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:15.888169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:15.888169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:15.888169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:15.888169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:15.889963 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 306/626] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:15.891329 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3355c-3fd7-4c16-b68a-4a0481323915 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:15.891624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3355c-3fd7-4c16-b68a-4a0481323915 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:15.891907 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3355c-3fd7-4c16-b68a-4a0481323915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:15.910988 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-e3d3355c-3fd7-4c16-b68a-4a0481323915 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:15.912241 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-e3d3355c-3fd7-4c16-b68a-4a0481323915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:15.969998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500f33a-4472-46ec-b4ce-e4173107869a None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:15.989377 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 307/627] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:27:15.989377 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa539136-136c-4ae6-8b7c-fd77fd59fd5f None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:15.989377 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa539136-136c-4ae6-8b7c-fd77fd59fd5f None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:15.989377 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa539136-136c-4ae6-8b7c-fd77fd59fd5f None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:15.989377 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 308/628] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:15.989377 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa539136-136c-4ae6-8b7c-fd77fd59fd5f None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:16.017389 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535da94-ca0b-4627-948e-033c18993ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:16.017985 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535da94-ca0b-4627-948e-033c18993ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:16.018268 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535da94-ca0b-4627-948e-033c18993ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:16.018580 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535da94-ca0b-4627-948e-033c18993ad3 None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:16.019440 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535da94-ca0b-4627-948e-033c18993ad3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:16.019743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535da94-ca0b-4627-948e-033c18993ad3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:16.021195 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535da94-ca0b-4627-948e-033c18993ad3 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:16.038330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 321/629] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Jun 27 18:27:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:16.039595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba149ccd-b45c-458f-a900-ceb27f451716 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:16.039882 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba149ccd-b45c-458f-a900-ceb27f451716 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:16.040162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba149ccd-b45c-458f-a900-ceb27f451716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:16.057825 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-ba149ccd-b45c-458f-a900-ceb27f451716 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:16.059052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ba149ccd-b45c-458f-a900-ceb27f451716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:16.066711 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:16.105860 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa539136-136c-4ae6-8b7c-fd77fd59fd5f None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:16.122714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:16.132683 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 322/630] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:27:16.133645 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba149ccd-b45c-458f-a900-ceb27f451716 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:16.144874 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f03d8e-a358-4d12-b769-ab6843b80a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:16.145601 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f03d8e-a358-4d12-b769-ab6843b80a88 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:16.145949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f03d8e-a358-4d12-b769-ab6843b80a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:16.146287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f03d8e-a358-4d12-b769-ab6843b80a88 None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:16.147288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f03d8e-a358-4d12-b769-ab6843b80a88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:16.147661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f03d8e-a358-4d12-b769-ab6843b80a88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:16.149243 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f03d8e-a358-4d12-b769-ab6843b80a88 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:16.163467 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 309/631] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:16.164459 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f03d8e-a358-4d12-b769-ab6843b80a88 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:16.252836 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba149ccd-b45c-458f-a900-ceb27f451716 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:16.276198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f03d8e-a358-4d12-b769-ab6843b80a88 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:16.290010 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800bb06-cfc3-4ac9-b879-fbe169462bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:16.290840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800bb06-cfc3-4ac9-b879-fbe169462bdd None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:16.291127 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800bb06-cfc3-4ac9-b879-fbe169462bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:16.291404 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800bb06-cfc3-4ac9-b879-fbe169462bdd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:16.301900 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6800bb06-cfc3-4ac9-b879-fbe169462bdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:16.303508 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:16.310210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-6800bb06-cfc3-4ac9-b879-fbe169462bdd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:16.310210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:16.310210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:16.310210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:16.310210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:16.310210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:16.310210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:16.310210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:16.310210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:16.310210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:16.310210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:16.310210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:16.310210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:16.310210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:16.310210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:16.310210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:16.311361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:16.311361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:16.311361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:16.311361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:16.311361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:16.311361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:16.311361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:16.311361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:16.311361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:16.311361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:16.311361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:16.311361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:16.311361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:16.311361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:16.311361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:16.311361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:16.312266 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:16.312266 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:16.312266 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:16.312266 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:16.312266 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:16.312266 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:16.312266 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:16.312266 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:16.312266 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:16.312266 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:16.312266 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:16.312266 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:16.312266 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 323/632] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:27:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:16.312266 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800bb06-cfc3-4ac9-b879-fbe169462bdd None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:16.316667 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076841e2-3592-4797-aa1a-a0cf9ab6a706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:16.317322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076841e2-3592-4797-aa1a-a0cf9ab6a706 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:16.317606 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076841e2-3592-4797-aa1a-a0cf9ab6a706 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:16.317882 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076841e2-3592-4797-aa1a-a0cf9ab6a706 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:16.372692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:16.386669 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076841e2-3592-4797-aa1a-a0cf9ab6a706 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:16.388174 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076841e2-3592-4797-aa1a-a0cf9ab6a706 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:16.420722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800bb06-cfc3-4ac9-b879-fbe169462bdd None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:16.456046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7f10a-514b-4a4f-8245-8d64cd941aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:16.456839 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7f10a-514b-4a4f-8245-8d64cd941aec None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:16.457124 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7f10a-514b-4a4f-8245-8d64cd941aec None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:16.457401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7f10a-514b-4a4f-8245-8d64cd941aec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:16.468492 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d7f10a-514b-4a4f-8245-8d64cd941aec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:16.476272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-08d7f10a-514b-4a4f-8245-8d64cd941aec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:16.476272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:16.476272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:16.476272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:16.476272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:16.476272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:16.476272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:16.476272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:16.476272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:16.476272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:16.476272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:16.476272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:16.476272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:16.476272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:16.476272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:16.476272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:16.477202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:16.477202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:16.477202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:16.477202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:16.477202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:16.477202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:16.477202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:16.477202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:16.477202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:16.477202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:16.477202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:16.477202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:16.477202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:16.477202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:16.477202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:16.477202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:16.478043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:16.478043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:16.478043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:16.478043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:16.478043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:16.478043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:16.478043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:16.478043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:16.478043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:16.478043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:16.478043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:16.478043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:16.478043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 324/633] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:27:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:16.478771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7f10a-514b-4a4f-8245-8d64cd941aec None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:16.494942 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 310/634] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:27:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:27:16.496668 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076841e2-3592-4797-aa1a-a0cf9ab6a706 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:16.561832 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:16.588322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7f10a-514b-4a4f-8245-8d64cd941aec None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:16.602277 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076841e2-3592-4797-aa1a-a0cf9ab6a706 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:16.603174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:16.625887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc0346-438b-482e-9342-34126403355a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:16.626793 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc0346-438b-482e-9342-34126403355a None admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:16.627075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc0346-438b-482e-9342-34126403355a None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:16.627349 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc0346-438b-482e-9342-34126403355a None admin] PATH_INFO: `/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/a1529c9a45434a19968e924797688f06/roles/c4bc6e72b9d647c5b2485d52d55f7dd4` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:16.639294 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494cc46-2539-4dee-9fae-8576c78b9c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:16.639954 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494cc46-2539-4dee-9fae-8576c78b9c97 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:16.640236 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494cc46-2539-4dee-9fae-8576c78b9c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:16.640511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494cc46-2539-4dee-9fae-8576c78b9c97 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:16.641250 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494cc46-2539-4dee-9fae-8576c78b9c97 None admin] RBAC: Adding query filter params (name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:16.641555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494cc46-2539-4dee-9fae-8576c78b9c97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:16.642685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc0346-438b-482e-9342-34126403355a None admin] RBAC: Authorizing `identity:create_grant(project_id=84ecf552f8364884b4aeff3428cb81e4, user_id=a1529c9a45434a19968e924797688f06, role_id=c4bc6e72b9d647c5b2485d52d55f7dd4)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:16.642957 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494cc46-2539-4dee-9fae-8576c78b9c97 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:16.644184 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc0346-438b-482e-9342-34126403355a None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:16.651808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 311/635] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Jun 27 18:27:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:16.652611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494cc46-2539-4dee-9fae-8576c78b9c97 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:16.722673 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 325/636] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:27:16 2022] PUT /identity/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/a1529c9a45434a19968e924797688f06/roles/c4bc6e72b9d647c5b2485d52d55f7dd4 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:27:16.724161 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc0346-438b-482e-9342-34126403355a None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:16.775830 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494cc46-2539-4dee-9fae-8576c78b9c97 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:16.819468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961afcd-6127-45c3-a9d8-05cdaee7f453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:16.820621 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961afcd-6127-45c3-a9d8-05cdaee7f453 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:16.821215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961afcd-6127-45c3-a9d8-05cdaee7f453 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:16.821806 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961afcd-6127-45c3-a9d8-05cdaee7f453 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:16.822873 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2961afcd-6127-45c3-a9d8-05cdaee7f453 None admin] RBAC: Adding query filter params (name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:16.823740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2961afcd-6127-45c3-a9d8-05cdaee7f453 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:16.825690 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2961afcd-6127-45c3-a9d8-05cdaee7f453 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:16.838834 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:16.840478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 312/637] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Jun 27 18:27:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:16.841478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961afcd-6127-45c3-a9d8-05cdaee7f453 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:16.868522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc0346-438b-482e-9342-34126403355a None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:16.907014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102454e-0660-4eb7-987d-721a0cda01bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:16.907810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102454e-0660-4eb7-987d-721a0cda01bf None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:16.908096 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102454e-0660-4eb7-987d-721a0cda01bf None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:16.908377 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102454e-0660-4eb7-987d-721a0cda01bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:16.920342 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102454e-0660-4eb7-987d-721a0cda01bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:16.921824 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102454e-0660-4eb7-987d-721a0cda01bf None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:16.940174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-5102454e-0660-4eb7-987d-721a0cda01bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:16.940174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:16.940174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:16.940174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:16.940174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:16.940174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:16.940174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:16.940174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:16.940174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:16.940174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:16.940174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:16.940174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:16.940174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:16.940174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:16.940174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:16.941139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:16.941139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:16.941139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:16.941139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:16.941139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:16.941139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:16.941139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:16.941139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:16.941139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:16.941139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:16.941139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:16.941139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:16.941139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:16.941139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:16.941139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:16.941139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:16.941139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:16.942020 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:16.942020 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:16.942020 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:16.942020 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:16.942020 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 326/638] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:16.942624 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c607e-2ccc-466b-997b-419b70c1e477 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:16.942899 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c607e-2ccc-466b-997b-419b70c1e477 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:16.943179 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c607e-2ccc-466b-997b-419b70c1e477 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:16.944569 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 327/639] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:16.946512 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c607e-2ccc-466b-997b-419b70c1e477 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:16.956402 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2961afcd-6127-45c3-a9d8-05cdaee7f453 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:16.967589 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:16.992245 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a51681-3b84-4fb0-b140-2be35b4bc893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:16.993039 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a51681-3b84-4fb0-b140-2be35b4bc893 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:16.993321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a51681-3b84-4fb0-b140-2be35b4bc893 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:16.993598 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a51681-3b84-4fb0-b140-2be35b4bc893 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:17.005415 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a51681-3b84-4fb0-b140-2be35b4bc893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:17.006912 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a51681-3b84-4fb0-b140-2be35b4bc893 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:17.024884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-b0a51681-3b84-4fb0-b140-2be35b4bc893 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:17.024884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:17.024884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:17.024884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:17.024884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:17.024884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:17.024884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:17.024884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:17.024884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:17.024884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:17.024884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:17.024884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:17.024884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:17.024884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:17.024884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:17.026167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:17.026167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:17.026167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:17.026167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:17.026167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:17.026167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:17.026167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:17.026167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:17.026167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:17.026167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:17.026167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:17.026167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:17.026167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:17.026167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:17.026167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:17.026167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:17.026167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:17.027677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:17.027677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:17.027677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:17.027677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:17.027677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 313/640] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:17.027677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a51681-3b84-4fb0-b140-2be35b4bc893 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:17.030505 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:17.064239 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c607e-2ccc-466b-997b-419b70c1e477 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:17.099312 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8c54b-4426-49b3-8e43-71cc7c860f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:17.100103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8c54b-4426-49b3-8e43-71cc7c860f28 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:17.100386 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8c54b-4426-49b3-8e43-71cc7c860f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:17.100661 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8c54b-4426-49b3-8e43-71cc7c860f28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:17.111324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d8c54b-4426-49b3-8e43-71cc7c860f28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:17.112770 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d8c54b-4426-49b3-8e43-71cc7c860f28 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:17.130909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-f9d8c54b-4426-49b3-8e43-71cc7c860f28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:17.130909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:17.130909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:17.130909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:17.130909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:17.130909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:17.130909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:17.130909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:17.130909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:17.130909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:17.130909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:17.130909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:17.130909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:17.130909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:17.130909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:17.132090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:17.132090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:17.132090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:17.132090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:17.132090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:17.132090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:17.132090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:17.132090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:17.132090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:17.132090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:17.132090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:17.132090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:17.132090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:17.132090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:17.132090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:17.132090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:17.132090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:17.133174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:17.133174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:17.133174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:17.133174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:17.133174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 328/641] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:17.133532 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a28bcf-dc2c-4782-aa46-b6ca7be42aea None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:17.133532 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a28bcf-dc2c-4782-aa46-b6ca7be42aea None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:17.133949 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a28bcf-dc2c-4782-aa46-b6ca7be42aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:17.141157 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a51681-3b84-4fb0-b140-2be35b4bc893 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:17.150797 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-51a28bcf-dc2c-4782-aa46-b6ca7be42aea None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:17.152013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-51a28bcf-dc2c-4782-aa46-b6ca7be42aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:17.177406 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e36fd-6a39-4ef4-b55a-25f28a1de991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:17.178233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e36fd-6a39-4ef4-b55a-25f28a1de991 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:17.178537 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e36fd-6a39-4ef4-b55a-25f28a1de991 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:17.178817 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e36fd-6a39-4ef4-b55a-25f28a1de991 None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:17.179537 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e36fd-6a39-4ef4-b55a-25f28a1de991 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:17.179845 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e36fd-6a39-4ef4-b55a-25f28a1de991 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:17.181213 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e36fd-6a39-4ef4-b55a-25f28a1de991 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:17.198052 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 314/642] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:17.199859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e36fd-6a39-4ef4-b55a-25f28a1de991 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:17.215166 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:17.223537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 329/643] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:27:17.224353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a28bcf-dc2c-4782-aa46-b6ca7be42aea None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:17.282345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:17.321559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e36fd-6a39-4ef4-b55a-25f28a1de991 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:17.338406 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a28bcf-dc2c-4782-aa46-b6ca7be42aea None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:17.357830 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add91068-91e4-46b6-a043-aa3abdad46eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:17.358491 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add91068-91e4-46b6-a043-aa3abdad46eb None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:17.358794 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add91068-91e4-46b6-a043-aa3abdad46eb None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:17.359069 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add91068-91e4-46b6-a043-aa3abdad46eb None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:17.359784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add91068-91e4-46b6-a043-aa3abdad46eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:17.360086 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add91068-91e4-46b6-a043-aa3abdad46eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:17.361455 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add91068-91e4-46b6-a043-aa3abdad46eb None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:17.377227 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 315/644] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:17.377227 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add91068-91e4-46b6-a043-aa3abdad46eb None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:17.379279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fab41-0bd0-42f0-a397-23d26de74d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:17.379947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fab41-0bd0-42f0-a397-23d26de74d54 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:17.380234 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fab41-0bd0-42f0-a397-23d26de74d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:17.380509 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fab41-0bd0-42f0-a397-23d26de74d54 None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:17.381240 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fab41-0bd0-42f0-a397-23d26de74d54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:17.381553 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fab41-0bd0-42f0-a397-23d26de74d54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:17.382984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fab41-0bd0-42f0-a397-23d26de74d54 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:17.399664 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 330/645] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:17.399664 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fab41-0bd0-42f0-a397-23d26de74d54 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:17.454903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:17.488520 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add91068-91e4-46b6-a043-aa3abdad46eb None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:17.504150 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fab41-0bd0-42f0-a397-23d26de74d54 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:17.511693 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:17.523341 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eeab1d-d6dc-4461-820a-896b091b0549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:17.524384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eeab1d-d6dc-4461-820a-896b091b0549 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:17.524668 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eeab1d-d6dc-4461-820a-896b091b0549 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:17.524942 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eeab1d-d6dc-4461-820a-896b091b0549 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:17.531983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eeab1d-d6dc-4461-820a-896b091b0549 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:17.533417 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eeab1d-d6dc-4461-820a-896b091b0549 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:17.540547 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56d00e-35d7-40bb-bf02-1db46743ef6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:17.541531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56d00e-35d7-40bb-bf02-1db46743ef6c None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:17.542518 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56d00e-35d7-40bb-bf02-1db46743ef6c None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:17.542664 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56d00e-35d7-40bb-bf02-1db46743ef6c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:17.550619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-40eeab1d-d6dc-4461-820a-896b091b0549 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:17.550619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:17.550619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:17.550619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:17.550619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:17.550619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:17.550619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:17.550619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:17.550619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:17.550619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:17.550619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:17.550619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:17.550619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:17.550619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:17.550619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:17.551756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:17.551756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:17.551756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:17.551756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:17.551756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:17.551756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:17.551756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:17.551756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:17.551756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:17.551756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:17.551756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:17.551756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:17.551756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:17.551756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:17.551756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:17.551756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:17.551756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:17.552871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:17.552871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:17.552871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:17.552871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:17.552871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:17.552871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:17.552871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:17.552871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:17.552871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 316/646] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:17.553423 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eeab1d-d6dc-4461-820a-896b091b0549 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:17.560672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa56d00e-35d7-40bb-bf02-1db46743ef6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:17.569523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-fa56d00e-35d7-40bb-bf02-1db46743ef6c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:17.569523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:17.569523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:17.569523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:17.569523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:17.569523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:17.569523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:17.569523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:17.569523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:17.569523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:17.569523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:17.569523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:17.569523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:17.569523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:17.569523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:17.569523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:17.570516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:17.570516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:17.570516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:17.570516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:17.570516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:17.570516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:17.570516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:17.570516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:17.570516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:17.570516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:17.570516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:17.570516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:17.570516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:17.570516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:17.570516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:17.570516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:17.571399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:17.571399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:17.571399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:17.571399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:17.571399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:17.571399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:17.571399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:17.571399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:17.571399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:17.571399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:17.571399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:17.571399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:17.571399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 331/647] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:27:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:17.572120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56d00e-35d7-40bb-bf02-1db46743ef6c None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:17.664388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eeab1d-d6dc-4461-820a-896b091b0549 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:17.678078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56d00e-35d7-40bb-bf02-1db46743ef6c None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:17.695602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:17.701770 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b34925-9e1b-4fc9-99a7-097e119a6a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:17.702585 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b34925-9e1b-4fc9-99a7-097e119a6a5f None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:17.702865 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b34925-9e1b-4fc9-99a7-097e119a6a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:17.703138 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b34925-9e1b-4fc9-99a7-097e119a6a5f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:17.710130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b34925-9e1b-4fc9-99a7-097e119a6a5f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:17.713211 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b34925-9e1b-4fc9-99a7-097e119a6a5f None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:17.715328 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f9c2c-0851-466f-8402-b5d770b7b5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:17.716115 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f9c2c-0851-466f-8402-b5d770b7b5ca None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:17.716399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f9c2c-0851-466f-8402-b5d770b7b5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:17.716678 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f9c2c-0851-466f-8402-b5d770b7b5ca None admin] PATH_INFO: `/v3/users/glance` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:17.724325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f9c2c-0851-466f-8402-b5d770b7b5ca None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:17.725862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f9c2c-0851-466f-8402-b5d770b7b5ca None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:17.726744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-73b34925-9e1b-4fc9-99a7-097e119a6a5f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 18:27:17.726744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:17.726744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:17.726744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:17.726744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:17.726744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:17.726744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:17.726744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:17.726744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:17.726744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:17.726744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:17.726744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:17.726744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:17.726744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:17.726744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:17.726744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:17.727662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:17.727662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:17.727662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:17.727662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:17.727662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:17.727662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:17.727662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:17.727662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:17.727662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:17.727662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:17.727662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:17.727662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:17.727662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:17.727662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:17.727662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:17.727662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:17.727662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:17.727662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:17.728617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:17.728617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:17.728617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:17.728617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:17.728617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:17.728617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:17.728617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:17.728617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:17.728617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 18:27:17.728617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:17.728617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 317/648] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:27:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:17.729248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b34925-9e1b-4fc9-99a7-097e119a6a5f None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:17.747729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-d80f9c2c-0851-466f-8402-b5d770b7b5ca None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 27 18:27:17.747729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:17.747729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:17.747729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:17.747729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:17.747729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:17.747729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:17.747729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:17.747729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:17.747729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:17.747729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:17.747729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:17.747729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:17.747729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:17.747729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:17.747729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:17.748732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:17.748732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:17.748732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:17.748732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:17.748732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:17.748732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:17.748732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:17.748732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:17.748732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:17.748732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:17.748732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:17.748732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:17.748732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:17.748732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:17.748732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:17.748732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:17.748732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:17.748732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:17.749751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:17.749751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:17.749751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:17.749751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:17.749751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:17.749751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:17.749751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:17.749751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:17.749751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 27 18:27:17.749751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:17.749751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 332/649] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:27:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:17.749751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f9c2c-0851-466f-8402-b5d770b7b5ca None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:17.782257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:17.841845 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b34925-9e1b-4fc9-99a7-097e119a6a5f None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:17.862283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f9c2c-0851-466f-8402-b5d770b7b5ca None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:17.880196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47705cb-7bbf-4525-bbaf-a163274d688b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:17.880850 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47705cb-7bbf-4525-bbaf-a163274d688b None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:17.881137 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47705cb-7bbf-4525-bbaf-a163274d688b None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:17.881415 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47705cb-7bbf-4525-bbaf-a163274d688b None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:17.882202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47705cb-7bbf-4525-bbaf-a163274d688b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:17.882543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47705cb-7bbf-4525-bbaf-a163274d688b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:17.883979 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47705cb-7bbf-4525-bbaf-a163274d688b None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:17.900535 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 318/650] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:17.900535 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47705cb-7bbf-4525-bbaf-a163274d688b None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:17.903035 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68b098-6ba7-4ebb-8689-a729cf258476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:17.903705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68b098-6ba7-4ebb-8689-a729cf258476 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:17.903992 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68b098-6ba7-4ebb-8689-a729cf258476 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:17.904268 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68b098-6ba7-4ebb-8689-a729cf258476 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:17.905019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a68b098-6ba7-4ebb-8689-a729cf258476 None admin] RBAC: Adding query filter params (name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:17.905325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a68b098-6ba7-4ebb-8689-a729cf258476 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:17.906741 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a68b098-6ba7-4ebb-8689-a729cf258476 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:17.919833 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 333/651] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Jun 27 18:27:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:17.919833 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68b098-6ba7-4ebb-8689-a729cf258476 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:17.958926 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:18.009021 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47705cb-7bbf-4525-bbaf-a163274d688b None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:18.012083 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:18.020286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68b098-6ba7-4ebb-8689-a729cf258476 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:18.045590 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a140cb-bc57-4902-97d2-87dbbbe5cf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:18.046243 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a140cb-bc57-4902-97d2-87dbbbe5cf0f None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:18.046557 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a140cb-bc57-4902-97d2-87dbbbe5cf0f None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:18.046832 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a140cb-bc57-4902-97d2-87dbbbe5cf0f None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:18.047661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a140cb-bc57-4902-97d2-87dbbbe5cf0f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:18.048120 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a140cb-bc57-4902-97d2-87dbbbe5cf0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:18.049363 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a140cb-bc57-4902-97d2-87dbbbe5cf0f None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:18.055667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc46f7-f286-4a3d-9bcd-b9f9369307af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:18.056537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc46f7-f286-4a3d-9bcd-b9f9369307af None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:18.056892 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc46f7-f286-4a3d-9bcd-b9f9369307af None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:18.057376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc46f7-f286-4a3d-9bcd-b9f9369307af None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:18.058353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc46f7-f286-4a3d-9bcd-b9f9369307af None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:18.058704 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc46f7-f286-4a3d-9bcd-b9f9369307af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:18.060140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc46f7-f286-4a3d-9bcd-b9f9369307af None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:18.071414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 319/652] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Jun 27 18:27:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:18.071414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a140cb-bc57-4902-97d2-87dbbbe5cf0f None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:18.089523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 334/653] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Jun 27 18:27:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:18.089523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc46f7-f286-4a3d-9bcd-b9f9369307af None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:18.187365 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a140cb-bc57-4902-97d2-87dbbbe5cf0f None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:18.193354 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:18.199718 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc46f7-f286-4a3d-9bcd-b9f9369307af None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:18.225550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af551923-6c9a-4429-8a63-28318fdc46eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:18.226198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af551923-6c9a-4429-8a63-28318fdc46eb None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:18.226510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af551923-6c9a-4429-8a63-28318fdc46eb None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:18.226788 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af551923-6c9a-4429-8a63-28318fdc46eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:18.227741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af551923-6c9a-4429-8a63-28318fdc46eb None admin] RBAC: Adding query filter params (role.id=c4bc6e72b9d647c5b2485d52d55f7dd4, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=1dddd4efb44c4152bb4151f7715f2223) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:18.228169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af551923-6c9a-4429-8a63-28318fdc46eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:18.229766 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af551923-6c9a-4429-8a63-28318fdc46eb None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:18.236064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110136c-667f-4866-9d46-bd9957be6d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:18.246664 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 320/654] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:27:18 2022] GET /identity/v3/role_assignments?user.id=1dddd4efb44c4152bb4151f7715f2223&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=c4bc6e72b9d647c5b2485d52d55f7dd4 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:18.246908 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110136c-667f-4866-9d46-bd9957be6d9d None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:18.246908 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110136c-667f-4866-9d46-bd9957be6d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:18.246908 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110136c-667f-4866-9d46-bd9957be6d9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:18.247169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af551923-6c9a-4429-8a63-28318fdc46eb None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:18.258073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110136c-667f-4866-9d46-bd9957be6d9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:18.259612 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110136c-667f-4866-9d46-bd9957be6d9d None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:18.281305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-0110136c-667f-4866-9d46-bd9957be6d9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:18.281305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:18.281305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:18.281305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:18.281305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:18.281305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:18.281305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:18.281305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:18.281305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:18.281305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:18.281305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:18.281305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:18.281305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:18.281305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:18.281305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:18.282269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:18.282269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:18.282269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:18.282269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:18.282269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:18.282269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:18.282269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:18.282269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:18.282269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:18.282269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:18.282269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:18.282269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:18.282269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:18.282269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:18.282269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:18.282269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:18.282269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:18.283242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:18.283242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:18.283242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:18.283242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:18.284862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:18.287428 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 335/655] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:18.288452 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110136c-667f-4866-9d46-bd9957be6d9d None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:18.392043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af551923-6c9a-4429-8a63-28318fdc46eb None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:18.413208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110136c-667f-4866-9d46-bd9957be6d9d None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:18.440272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080c5f5-c7ee-4016-aad9-7f731ceecb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:18.441450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080c5f5-c7ee-4016-aad9-7f731ceecb0a None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:18.441736 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080c5f5-c7ee-4016-aad9-7f731ceecb0a None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:18.442178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080c5f5-c7ee-4016-aad9-7f731ceecb0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:18.454356 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff72f98-5092-4d93-9729-98de8558449c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:18.455050 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff72f98-5092-4d93-9729-98de8558449c None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:18.455334 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff72f98-5092-4d93-9729-98de8558449c None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:18.455613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff72f98-5092-4d93-9729-98de8558449c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:18.479503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:18.537370 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080c5f5-c7ee-4016-aad9-7f731ceecb0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:18.539217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080c5f5-c7ee-4016-aad9-7f731ceecb0a None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:18.544220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff72f98-5092-4d93-9729-98de8558449c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:18.545689 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff72f98-5092-4d93-9729-98de8558449c None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:18.577216 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:18.666119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 321/656] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:27:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:27:18.667393 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080c5f5-c7ee-4016-aad9-7f731ceecb0a None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:18.679533 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 336/657] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:27:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:27:18.680819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff72f98-5092-4d93-9729-98de8558449c None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:18.737465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:18.783729 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080c5f5-c7ee-4016-aad9-7f731ceecb0a None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:18.794207 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff72f98-5092-4d93-9729-98de8558449c None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:18.810652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:18.821247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0652ca-6e31-4bd0-bb57-b02214f2c8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:18.824692 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0652ca-6e31-4bd0-bb57-b02214f2c8a4 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:18.824692 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0652ca-6e31-4bd0-bb57-b02214f2c8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:18.824692 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0652ca-6e31-4bd0-bb57-b02214f2c8a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:18.824692 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0652ca-6e31-4bd0-bb57-b02214f2c8a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:18.824692 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0652ca-6e31-4bd0-bb57-b02214f2c8a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:18.825442 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0652ca-6e31-4bd0-bb57-b02214f2c8a4 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:18.832269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61728133-e9e8-4961-83d2-474dbd39245a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:18.833274 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61728133-e9e8-4961-83d2-474dbd39245a None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:18.833716 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61728133-e9e8-4961-83d2-474dbd39245a None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:18.833982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61728133-e9e8-4961-83d2-474dbd39245a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:18.837435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 322/658] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:18.838497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0652ca-6e31-4bd0-bb57-b02214f2c8a4 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:18.846933 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61728133-e9e8-4961-83d2-474dbd39245a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:18.848808 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61728133-e9e8-4961-83d2-474dbd39245a None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:18.868750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-61728133-e9e8-4961-83d2-474dbd39245a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:18.868750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:18.868750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:18.868750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:18.868750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:18.868750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:18.868750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:18.868750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:18.868750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:18.868750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:18.868750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:18.868750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:18.868750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:18.868750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:18.868750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:18.871869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:18.871869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:18.871869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:18.871869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:18.871869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:18.871869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:18.871869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:18.871869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:18.871869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:18.871869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:18.871869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:18.871869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:18.871869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:18.871869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:18.871869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:18.871869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:18.871869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:18.872760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:18.872760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:18.872760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:18.872760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:18.872760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 337/659] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:18.872760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f6758-b510-4b4e-b0f8-d40ccb3a1ab6 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:18.873085 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f6758-b510-4b4e-b0f8-d40ccb3a1ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:18.873085 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f6758-b510-4b4e-b0f8-d40ccb3a1ab6 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:18.878411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 338/660] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:18.879167 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f6758-b510-4b4e-b0f8-d40ccb3a1ab6 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:18.956176 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0652ca-6e31-4bd0-bb57-b02214f2c8a4 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:18.982544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:18.986139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f6758-b510-4b4e-b0f8-d40ccb3a1ab6 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:18.996680 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38400a4-e161-48e9-bd97-befb0923c16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:18.996680 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38400a4-e161-48e9-bd97-befb0923c16c None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:18.996680 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38400a4-e161-48e9-bd97-befb0923c16c None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:18.996680 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38400a4-e161-48e9-bd97-befb0923c16c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:19.007343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38400a4-e161-48e9-bd97-befb0923c16c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:19.009101 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38400a4-e161-48e9-bd97-befb0923c16c None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:19.021113 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcea9e-868d-4121-8eef-2ef0909f1944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:19.021897 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcea9e-868d-4121-8eef-2ef0909f1944 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:19.022178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcea9e-868d-4121-8eef-2ef0909f1944 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:19.022476 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcea9e-868d-4121-8eef-2ef0909f1944 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:19.030112 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-c38400a4-e161-48e9-bd97-befb0923c16c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:19.030112 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:19.030112 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:19.030112 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:19.030112 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:19.030112 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:19.030112 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:19.030112 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:19.030112 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:19.030112 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:19.030112 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:19.030112 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:19.030112 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:19.030112 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:19.030112 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:19.031221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:19.031221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:19.031221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:19.031221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:19.031221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:19.031221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:19.031221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:19.031221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:19.031221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:19.031221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:19.031221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:19.031221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:19.031221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:19.031221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:19.031221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:19.031221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:19.031221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:19.032355 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:19.032355 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:19.032355 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:19.032355 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:19.033350 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bcea9e-868d-4121-8eef-2ef0909f1944 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:19.034815 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bcea9e-868d-4121-8eef-2ef0909f1944 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:19.035896 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 323/661] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:19.036457 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38400a4-e161-48e9-bd97-befb0923c16c None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:19.049947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-74bcea9e-868d-4121-8eef-2ef0909f1944 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 18:27:19.049947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:19.049947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:19.049947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:19.049947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:19.049947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:19.049947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:19.049947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:19.049947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:19.049947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:19.049947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:19.049947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:19.049947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:19.049947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:19.049947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:19.049947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:19.050959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:19.050959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:19.050959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:19.050959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:19.050959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:19.050959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:19.050959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:19.050959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:19.050959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:19.050959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:19.050959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:19.050959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:19.050959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:19.050959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:19.050959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:19.050959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:19.050959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:19.050959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:19.056753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:19.056753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:19.056753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:19.056753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:19.056753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:19.056753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:19.056753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:19.056753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:19.056753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 18:27:19.056753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:19.056753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 339/662] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:27:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:19.056753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647f936-c357-428d-9186-2f052db3e794 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:19.056753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647f936-c357-428d-9186-2f052db3e794 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:19.057644 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647f936-c357-428d-9186-2f052db3e794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:19.070996 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-c647f936-c357-428d-9186-2f052db3e794 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:19.072177 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c647f936-c357-428d-9186-2f052db3e794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:19.083925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:19.146902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 340/663] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:27:19.147738 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647f936-c357-428d-9186-2f052db3e794 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:19.160857 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38400a4-e161-48e9-bd97-befb0923c16c None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:19.196663 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a6958-f61d-41dd-8e4f-5593d6354495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:19.197366 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a6958-f61d-41dd-8e4f-5593d6354495 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:19.197601 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a6958-f61d-41dd-8e4f-5593d6354495 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:19.198073 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a6958-f61d-41dd-8e4f-5593d6354495 None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:19.199399 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a6958-f61d-41dd-8e4f-5593d6354495 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:19.199706 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a6958-f61d-41dd-8e4f-5593d6354495 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:19.201060 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a6958-f61d-41dd-8e4f-5593d6354495 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:19.217212 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 324/664] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:19.217212 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a6958-f61d-41dd-8e4f-5593d6354495 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:19.257059 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:19.267745 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647f936-c357-428d-9186-2f052db3e794 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:19.300996 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ec264-7b22-4e16-a266-5442315c44c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:19.301653 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ec264-7b22-4e16-a266-5442315c44c9 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:19.301936 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ec264-7b22-4e16-a266-5442315c44c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:19.302211 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ec264-7b22-4e16-a266-5442315c44c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:19.302969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ec264-7b22-4e16-a266-5442315c44c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:19.303275 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ec264-7b22-4e16-a266-5442315c44c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:19.304646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ec264-7b22-4e16-a266-5442315c44c9 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:19.320450 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 341/665] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:19.320450 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ec264-7b22-4e16-a266-5442315c44c9 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:19.321761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:19.328537 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a6958-f61d-41dd-8e4f-5593d6354495 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:19.367262 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901912c9-da5c-42fd-80da-c779f9191184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:19.368042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901912c9-da5c-42fd-80da-c779f9191184 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:19.368362 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901912c9-da5c-42fd-80da-c779f9191184 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:19.368648 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901912c9-da5c-42fd-80da-c779f9191184 None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:19.369482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901912c9-da5c-42fd-80da-c779f9191184 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:19.369792 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901912c9-da5c-42fd-80da-c779f9191184 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:19.372110 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901912c9-da5c-42fd-80da-c779f9191184 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:19.390604 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 325/666] 10.222.0.15 () {56 vars in 1139 bytes} [Mon Jun 27 18:27:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:19.396402 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901912c9-da5c-42fd-80da-c779f9191184 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:19.424484 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ec264-7b22-4e16-a266-5442315c44c9 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:19.459659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2733f52-e87e-4240-963b-16c31eb15885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:19.460441 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2733f52-e87e-4240-963b-16c31eb15885 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:19.460721 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2733f52-e87e-4240-963b-16c31eb15885 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:19.460993 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2733f52-e87e-4240-963b-16c31eb15885 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:19.471926 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2733f52-e87e-4240-963b-16c31eb15885 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:19.479513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-a2733f52-e87e-4240-963b-16c31eb15885 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:19.479513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:19.479513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:19.479513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:19.479513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:19.479513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:19.479513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:19.479513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:19.479513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:19.479513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:19.479513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:19.479513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:19.479513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:19.479513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:19.479513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:19.479513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:19.480469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:19.480469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:19.480469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:19.480469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:19.480469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:19.480469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:19.480469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:19.480469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:19.480469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:19.480469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:19.480469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:19.480469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:19.480469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:19.480469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:19.480469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:19.480469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:19.481330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:19.481330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:19.481330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:19.481330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:19.481330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:19.481330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:19.481330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:19.481330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:19.481330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:19.481330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:19.481330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:19.481330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:19.481330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 342/667] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:27:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:19.482968 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2733f52-e87e-4240-963b-16c31eb15885 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:19.496487 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:19.505784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901912c9-da5c-42fd-80da-c779f9191184 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:19.547292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfe18f-7abf-44bb-9ffb-602f412bb440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:19.548334 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfe18f-7abf-44bb-9ffb-602f412bb440 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:19.548770 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfe18f-7abf-44bb-9ffb-602f412bb440 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:19.549187 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfe18f-7abf-44bb-9ffb-602f412bb440 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:19.556487 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bfe18f-7abf-44bb-9ffb-602f412bb440 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:19.558556 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bfe18f-7abf-44bb-9ffb-602f412bb440 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:19.572368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-c6bfe18f-7abf-44bb-9ffb-602f412bb440 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:19.572368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:19.572368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:19.572368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:19.572368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:19.572368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:19.572368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:19.572368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:19.572368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:19.572368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:19.572368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:19.572368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:19.572368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:19.572368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:19.572368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:19.573982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:19.573982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:19.573982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:19.573982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:19.573982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:19.573982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:19.573982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:19.573982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:19.573982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:19.573982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:19.573982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:19.573982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:19.573982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:19.573982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:19.573982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:19.573982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:19.573982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:19.575333 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:19.575333 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:19.575333 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:19.575333 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:19.575333 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:19.575333 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:19.575333 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:19.575333 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:19.578853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 326/668] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:19.579692 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfe18f-7abf-44bb-9ffb-602f412bb440 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:19.595079 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2733f52-e87e-4240-963b-16c31eb15885 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:19.596941 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:19.635015 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b19d2-5c73-45a0-8978-cfea9295df91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:19.636322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b19d2-5c73-45a0-8978-cfea9295df91 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:19.636610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b19d2-5c73-45a0-8978-cfea9295df91 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:19.636883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b19d2-5c73-45a0-8978-cfea9295df91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:19.643722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b19d2-5c73-45a0-8978-cfea9295df91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:19.645225 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b19d2-5c73-45a0-8978-cfea9295df91 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:19.660014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-737b19d2-5c73-45a0-8978-cfea9295df91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:19.660014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:19.660014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:19.660014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:19.660014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:19.660014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:19.660014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:19.660014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:19.660014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:19.660014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:19.660014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:19.660014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:19.660014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:19.660014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:19.660014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:19.661103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:19.661103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:19.661103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:19.661103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:19.661103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:19.661103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:19.661103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:19.661103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:19.661103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:19.661103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:19.661103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:19.661103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:19.661103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:19.661103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:19.661103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:19.661103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:19.661103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:19.662002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:19.662002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:19.662002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:19.662002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:19.662002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:19.662002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:19.662002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:19.662002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:19.662002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 343/669] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:19.662002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b19d2-5c73-45a0-8978-cfea9295df91 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:19.691862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfe18f-7abf-44bb-9ffb-602f412bb440 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:19.725660 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9d85e-ef8e-4479-96de-08a409e9cbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:19.726294 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9d85e-ef8e-4479-96de-08a409e9cbae None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:19.726603 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9d85e-ef8e-4479-96de-08a409e9cbae None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:19.726875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9d85e-ef8e-4479-96de-08a409e9cbae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:19.735499 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:19.768985 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b19d2-5c73-45a0-8978-cfea9295df91 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:19.800794 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b9d85e-ef8e-4479-96de-08a409e9cbae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:19.802397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b9d85e-ef8e-4479-96de-08a409e9cbae None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:19.804032 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf8d8b-e27b-44e5-8586-021ac7236679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:19.804685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf8d8b-e27b-44e5-8586-021ac7236679 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:19.804964 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf8d8b-e27b-44e5-8586-021ac7236679 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:19.805241 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf8d8b-e27b-44e5-8586-021ac7236679 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:19.805983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf8d8b-e27b-44e5-8586-021ac7236679 None admin] RBAC: Adding query filter params (name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:19.806287 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf8d8b-e27b-44e5-8586-021ac7236679 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:19.808237 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf8d8b-e27b-44e5-8586-021ac7236679 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:19.820928 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 344/670] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Jun 27 18:27:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:19.820928 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf8d8b-e27b-44e5-8586-021ac7236679 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:19.823801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:19.918121 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 327/671] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:27:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:27:19.918121 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9d85e-ef8e-4479-96de-08a409e9cbae None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:19.927350 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf8d8b-e27b-44e5-8586-021ac7236679 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:19.956958 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:19.961819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990ff3b-a659-449c-8b7a-1b171dfa3abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:19.962652 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990ff3b-a659-449c-8b7a-1b171dfa3abd None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:19.963003 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990ff3b-a659-449c-8b7a-1b171dfa3abd None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:19.963289 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990ff3b-a659-449c-8b7a-1b171dfa3abd None admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:19.964794 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990ff3b-a659-449c-8b7a-1b171dfa3abd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:19.965106 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990ff3b-a659-449c-8b7a-1b171dfa3abd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:19.966565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990ff3b-a659-449c-8b7a-1b171dfa3abd None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:19.978309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 345/672] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:19.979137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990ff3b-a659-449c-8b7a-1b171dfa3abd None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:20.032127 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9d85e-ef8e-4479-96de-08a409e9cbae None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:20.048684 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:20.069371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af73565-d09d-4a62-bc69-216c68ad899d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:20.070327 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af73565-d09d-4a62-bc69-216c68ad899d None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:20.070735 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af73565-d09d-4a62-bc69-216c68ad899d None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:20.071081 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af73565-d09d-4a62-bc69-216c68ad899d None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:20.072043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af73565-d09d-4a62-bc69-216c68ad899d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:20.072310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af73565-d09d-4a62-bc69-216c68ad899d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:20.074706 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af73565-d09d-4a62-bc69-216c68ad899d None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:20.082534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990ff3b-a659-449c-8b7a-1b171dfa3abd None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:20.094223 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 328/673] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:20.094223 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af73565-d09d-4a62-bc69-216c68ad899d None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:20.124435 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c40bd-19a7-4685-b8bb-46ffa858bd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:20.125482 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c40bd-19a7-4685-b8bb-46ffa858bd36 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:20.125923 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c40bd-19a7-4685-b8bb-46ffa858bd36 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:20.126354 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c40bd-19a7-4685-b8bb-46ffa858bd36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:20.137800 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c40bd-19a7-4685-b8bb-46ffa858bd36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:20.139106 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c40bd-19a7-4685-b8bb-46ffa858bd36 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:20.158639 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-bb6c40bd-19a7-4685-b8bb-46ffa858bd36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:20.158639 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:20.158639 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:20.158639 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:20.158639 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:20.158639 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:20.158639 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:20.158639 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:20.158639 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:20.158639 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:20.158639 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:20.158639 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:20.158639 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:20.158639 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:20.158639 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:20.159591 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:20.159591 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:20.159591 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:20.159591 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:20.159591 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:20.159591 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:20.159591 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:20.159591 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:20.159591 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:20.159591 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:20.159591 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:20.159591 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:20.159591 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:20.159591 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:20.159591 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:20.159591 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:20.159591 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:20.160485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:20.160485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:20.160485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:20.160485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:20.160485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 346/674] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:20.160766 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c40bd-19a7-4685-b8bb-46ffa858bd36 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:20.204253 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af73565-d09d-4a62-bc69-216c68ad899d None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:20.210941 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:20.241095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869074c3-92f9-479b-be1c-c60709dc94fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:20.241881 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869074c3-92f9-479b-be1c-c60709dc94fe None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:20.242165 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869074c3-92f9-479b-be1c-c60709dc94fe None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:20.242468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869074c3-92f9-479b-be1c-c60709dc94fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:20.253555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869074c3-92f9-479b-be1c-c60709dc94fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:20.255031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869074c3-92f9-479b-be1c-c60709dc94fe None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:20.270364 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c40bd-19a7-4685-b8bb-46ffa858bd36 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:20.273319 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-869074c3-92f9-479b-be1c-c60709dc94fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:20.273319 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:20.273319 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:20.273319 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:20.273319 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:20.273319 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:20.273319 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:20.273319 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:20.273319 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:20.273319 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:20.273319 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:20.273319 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:20.273319 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:20.273319 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:20.273319 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:20.281636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:20.281636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:20.281636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:20.281636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:20.281636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:20.281636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:20.281636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:20.281636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:20.281636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:20.281636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:20.281636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:20.281636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:20.281636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:20.281636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:20.281636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:20.281636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:20.281636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:20.282611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:20.282611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:20.282611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:20.282611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:20.282611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 329/675] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:20.282611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869074c3-92f9-479b-be1c-c60709dc94fe None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:20.309742 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:20.309956 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a845c-2044-4adb-b3fa-7bada9f4fc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:20.310598 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a845c-2044-4adb-b3fa-7bada9f4fc9e None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:20.310878 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a845c-2044-4adb-b3fa-7bada9f4fc9e None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:20.311153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a845c-2044-4adb-b3fa-7bada9f4fc9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:20.311984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547a845c-2044-4adb-b3fa-7bada9f4fc9e None admin] RBAC: Adding query filter params (role.id=c4bc6e72b9d647c5b2485d52d55f7dd4, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=c4f6f8e0a0cb4c9a9dd0d99fc0022235) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:20.312287 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547a845c-2044-4adb-b3fa-7bada9f4fc9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:20.313725 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547a845c-2044-4adb-b3fa-7bada9f4fc9e None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:20.330499 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 347/676] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:27:20 2022] GET /identity/v3/role_assignments?user.id=c4f6f8e0a0cb4c9a9dd0d99fc0022235&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=c4bc6e72b9d647c5b2485d52d55f7dd4 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:20.330499 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a845c-2044-4adb-b3fa-7bada9f4fc9e None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:20.428070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869074c3-92f9-479b-be1c-c60709dc94fe None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:20.453761 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a845c-2044-4adb-b3fa-7bada9f4fc9e None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:20.465194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:20.479801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91adee97-27b4-4208-9d2e-dcff0742c435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:20.480811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adee97-27b4-4208-9d2e-dcff0742c435 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:20.481100 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adee97-27b4-4208-9d2e-dcff0742c435 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:20.481904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adee97-27b4-4208-9d2e-dcff0742c435 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:20.482972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91adee97-27b4-4208-9d2e-dcff0742c435 None admin] RBAC: Adding query filter params (role.id=c4bc6e72b9d647c5b2485d52d55f7dd4, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=75e75e5ae3ca45d0b503f6e52f399558) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:20.483296 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91adee97-27b4-4208-9d2e-dcff0742c435 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:20.485220 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91adee97-27b4-4208-9d2e-dcff0742c435 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:20.493541 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 330/677] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:27:20 2022] GET /identity/v3/role_assignments?user.id=75e75e5ae3ca45d0b503f6e52f399558&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=c4bc6e72b9d647c5b2485d52d55f7dd4 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:20.493996 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91adee97-27b4-4208-9d2e-dcff0742c435 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:20.496187 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c46834-4851-4acc-ae44-18283f941e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:20.497162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c46834-4851-4acc-ae44-18283f941e5b None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:20.497667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c46834-4851-4acc-ae44-18283f941e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:20.497948 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c46834-4851-4acc-ae44-18283f941e5b None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:20.498739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c46834-4851-4acc-ae44-18283f941e5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:20.499050 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c46834-4851-4acc-ae44-18283f941e5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:20.501714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c46834-4851-4acc-ae44-18283f941e5b None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:20.517475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 348/678] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:20.519911 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090e205-c180-47a2-9d45-c76ed49ceba4 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:20.520478 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090e205-c180-47a2-9d45-c76ed49ceba4 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:20.520941 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090e205-c180-47a2-9d45-c76ed49ceba4 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:20.523675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 349/679] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:20.525115 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090e205-c180-47a2-9d45-c76ed49ceba4 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:20.631132 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:20.644299 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91adee97-27b4-4208-9d2e-dcff0742c435 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:20.672409 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090e205-c180-47a2-9d45-c76ed49ceba4 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:20.697159 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740b510-e230-4254-983f-ef67a9ba821a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:20.697833 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740b510-e230-4254-983f-ef67a9ba821a None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:20.698533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740b510-e230-4254-983f-ef67a9ba821a None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:20.698751 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740b510-e230-4254-983f-ef67a9ba821a None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:20.699492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740b510-e230-4254-983f-ef67a9ba821a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:20.699812 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740b510-e230-4254-983f-ef67a9ba821a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:20.701185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740b510-e230-4254-983f-ef67a9ba821a None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:20.712791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 331/680] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:20.716294 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23a90a-03fa-4b51-a895-17a9edef0698 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:20.716294 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23a90a-03fa-4b51-a895-17a9edef0698 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:20.716294 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23a90a-03fa-4b51-a895-17a9edef0698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:20.725398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b784c-be57-49ec-b508-2c61cce40011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:20.726565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b784c-be57-49ec-b508-2c61cce40011 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:20.727074 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b784c-be57-49ec-b508-2c61cce40011 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:20.727552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b784c-be57-49ec-b508-2c61cce40011 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:20.737562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b784c-be57-49ec-b508-2c61cce40011 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:20.739830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b784c-be57-49ec-b508-2c61cce40011 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:20.740068 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-bf23a90a-03fa-4b51-a895-17a9edef0698 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:20.741285 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-bf23a90a-03fa-4b51-a895-17a9edef0698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:20.759019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-7d1b784c-be57-49ec-b508-2c61cce40011 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 18:27:20.759019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:20.759019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:20.759019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:20.759019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:20.759019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:20.759019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:20.759019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:20.759019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:20.759019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:20.759019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:20.759019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:20.759019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:20.759019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:20.759019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:20.759019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:20.760810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:20.760810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:20.760810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:20.760810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:20.760810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:20.760810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:20.760810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:20.760810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:20.760810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:20.760810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:20.760810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:20.760810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:20.760810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:20.760810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:20.760810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:20.760810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:20.760810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:20.760810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:20.761916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:20.761916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:20.761916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:20.761916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:20.761916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:20.761916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:20.761916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:20.761916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:20.761916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 18:27:20.761916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:20.761916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 350/681] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:27:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:20.761916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b784c-be57-49ec-b508-2c61cce40011 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:20.799585 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:20.832905 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 332/682] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:27:20.833757 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23a90a-03fa-4b51-a895-17a9edef0698 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:20.877333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b784c-be57-49ec-b508-2c61cce40011 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:20.914370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ca934-c6a6-4881-8301-ecf1dadb55f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:20.915125 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ca934-c6a6-4881-8301-ecf1dadb55f0 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:20.915509 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ca934-c6a6-4881-8301-ecf1dadb55f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:20.915792 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ca934-c6a6-4881-8301-ecf1dadb55f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:20.923164 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2ca934-c6a6-4881-8301-ecf1dadb55f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:20.924793 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2ca934-c6a6-4881-8301-ecf1dadb55f0 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:20.940541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-8e2ca934-c6a6-4881-8301-ecf1dadb55f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:20.940541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:20.940541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:20.940541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:20.940541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:20.940541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:20.940541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:20.940541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:20.940541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:20.940541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:20.940541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:20.940541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:20.940541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:20.940541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:20.940541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:20.941513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:20.941513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:20.941513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:20.941513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:20.941513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:20.941513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:20.941513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:20.941513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:20.941513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:20.941513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:20.941513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:20.941513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:20.941513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:20.941513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:20.941513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:20.941513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:20.941513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:20.942438 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:20.942438 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:20.942438 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:20.942438 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:20.942438 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:20.942438 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:20.942438 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:20.942438 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:20.946552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 351/683] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:20.947380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ca934-c6a6-4881-8301-ecf1dadb55f0 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:20.948347 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:20.969528 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23a90a-03fa-4b51-a895-17a9edef0698 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:21.008202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a3486-84f3-47a8-96b7-73e43b03ceef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:21.008869 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a3486-84f3-47a8-96b7-73e43b03ceef None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:21.009150 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a3486-84f3-47a8-96b7-73e43b03ceef None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:21.009421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a3486-84f3-47a8-96b7-73e43b03ceef None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:21.010412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a3486-84f3-47a8-96b7-73e43b03ceef None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:21.010748 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a3486-84f3-47a8-96b7-73e43b03ceef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:21.012504 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a3486-84f3-47a8-96b7-73e43b03ceef None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:21.036172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 333/684] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Jun 27 18:27:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:21.036926 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a3486-84f3-47a8-96b7-73e43b03ceef None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:21.046015 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:21.059724 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ca934-c6a6-4881-8301-ecf1dadb55f0 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:21.098995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea783732-2c68-49aa-b368-f64c90a7ace2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:21.100127 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea783732-2c68-49aa-b368-f64c90a7ace2 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:21.100640 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea783732-2c68-49aa-b368-f64c90a7ace2 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:21.101088 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea783732-2c68-49aa-b368-f64c90a7ace2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:21.113372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea783732-2c68-49aa-b368-f64c90a7ace2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:21.121838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-ea783732-2c68-49aa-b368-f64c90a7ace2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:27:21.121838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:21.121838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:21.121838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:21.121838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:21.121838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:21.121838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:21.121838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:21.121838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:21.121838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:21.121838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:21.121838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:21.121838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:21.121838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:21.121838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:21.121838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:21.123041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:21.123041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:21.123041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:21.123041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:21.123041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:21.123041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:21.123041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:21.123041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:21.123041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:21.123041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:21.123041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:21.123041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:21.123041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:21.123041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:21.123041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:21.123041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:21.123990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:21.123990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:21.123990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:21.123990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:21.123990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:21.123990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:21.123990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:21.123990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:21.123990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:21.123990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:21.123990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:27:21.123990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:21.126061 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 352/685] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:27:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:21.127100 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea783732-2c68-49aa-b368-f64c90a7ace2 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:21.160484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a3486-84f3-47a8-96b7-73e43b03ceef None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:21.200927 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da7025-4df4-42b5-b987-1f6a84f5e26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:21.201487 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da7025-4df4-42b5-b987-1f6a84f5e26b None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:21.201845 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da7025-4df4-42b5-b987-1f6a84f5e26b None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:21.202217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da7025-4df4-42b5-b987-1f6a84f5e26b None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:21.203109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da7025-4df4-42b5-b987-1f6a84f5e26b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:21.203487 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da7025-4df4-42b5-b987-1f6a84f5e26b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:21.205157 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da7025-4df4-42b5-b987-1f6a84f5e26b None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:21.214503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:21.220711 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 334/686] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:21.222021 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da7025-4df4-42b5-b987-1f6a84f5e26b None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:21.253887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea783732-2c68-49aa-b368-f64c90a7ace2 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:21.289834 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc18cd-f422-4ea4-8b81-7c42dfafbaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:21.290971 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc18cd-f422-4ea4-8b81-7c42dfafbaa1 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:21.291484 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc18cd-f422-4ea4-8b81-7c42dfafbaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:21.292474 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc18cd-f422-4ea4-8b81-7c42dfafbaa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:21.312202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:21.334678 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da7025-4df4-42b5-b987-1f6a84f5e26b None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:21.368565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc18cd-f422-4ea4-8b81-7c42dfafbaa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:21.370714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc18cd-f422-4ea4-8b81-7c42dfafbaa1 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:21.372721 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195fa20-e6c1-4c4c-a8cc-33198798482b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:21.373324 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195fa20-e6c1-4c4c-a8cc-33198798482b None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:21.373616 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195fa20-e6c1-4c4c-a8cc-33198798482b None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:21.373898 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f195fa20-e6c1-4c4c-a8cc-33198798482b None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:21.375070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195fa20-e6c1-4c4c-a8cc-33198798482b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:21.375205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195fa20-e6c1-4c4c-a8cc-33198798482b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:21.376465 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f195fa20-e6c1-4c4c-a8cc-33198798482b None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:21.390613 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 335/687] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:27:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:21.390613 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195fa20-e6c1-4c4c-a8cc-33198798482b None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:21.452611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:21.489140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 353/688] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:27:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:27:21.490732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc18cd-f422-4ea4-8b81-7c42dfafbaa1 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:21.496158 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f195fa20-e6c1-4c4c-a8cc-33198798482b None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:21.538992 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cff7cd-bb89-4fbd-a3dc-5b3ceb1f4ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:21.540390 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cff7cd-bb89-4fbd-a3dc-5b3ceb1f4ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:21.541317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cff7cd-bb89-4fbd-a3dc-5b3ceb1f4ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:21.541851 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cff7cd-bb89-4fbd-a3dc-5b3ceb1f4ee6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:21.543117 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cff7cd-bb89-4fbd-a3dc-5b3ceb1f4ee6 None admin] RBAC: Adding query filter params (role.id=c4bc6e72b9d647c5b2485d52d55f7dd4, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=dbe621916dbe4f1a8686097fe2bdf762) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:21.543581 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cff7cd-bb89-4fbd-a3dc-5b3ceb1f4ee6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:21.546016 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cff7cd-bb89-4fbd-a3dc-5b3ceb1f4ee6 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:21.547784 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:21.555159 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 336/689] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:27:21 2022] GET /identity/v3/role_assignments?user.id=dbe621916dbe4f1a8686097fe2bdf762&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=c4bc6e72b9d647c5b2485d52d55f7dd4 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:21.556514 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cff7cd-bb89-4fbd-a3dc-5b3ceb1f4ee6 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:21.610656 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc18cd-f422-4ea4-8b81-7c42dfafbaa1 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:21.648875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a93d9f-7b34-48fa-84d1-9b6198c0dfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:21.649821 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a93d9f-7b34-48fa-84d1-9b6198c0dfdd None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:21.650201 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a93d9f-7b34-48fa-84d1-9b6198c0dfdd None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:21.650700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a93d9f-7b34-48fa-84d1-9b6198c0dfdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:21.662926 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a93d9f-7b34-48fa-84d1-9b6198c0dfdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:21.664653 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a93d9f-7b34-48fa-84d1-9b6198c0dfdd None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:21.685611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-66a93d9f-7b34-48fa-84d1-9b6198c0dfdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:21.685611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:21.685611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:21.685611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:21.685611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:21.685611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:21.685611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:21.685611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:21.685611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:21.685611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:21.685611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:21.685611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:21.685611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:21.685611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:21.685611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:21.686876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:21.686876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:21.686876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:21.686876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:21.686876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:21.686876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:21.686876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:21.686876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:21.686876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:21.686876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:21.686876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:21.686876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:21.686876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:21.686876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:21.686876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:21.686876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:21.686876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:21.687811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:21.687811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:21.687811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:21.687811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:21.688033 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 354/690] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:21.693358 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a93d9f-7b34-48fa-84d1-9b6198c0dfdd None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:21.706300 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cff7cd-bb89-4fbd-a3dc-5b3ceb1f4ee6 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:21.736186 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:21.757604 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aec775-d562-4a41-b323-9d166c26a8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:21.758811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aec775-d562-4a41-b323-9d166c26a8ca None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:21.760309 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aec775-d562-4a41-b323-9d166c26a8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:21.760930 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aec775-d562-4a41-b323-9d166c26a8ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:21.779184 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aec775-d562-4a41-b323-9d166c26a8ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:21.782005 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aec775-d562-4a41-b323-9d166c26a8ca None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:21.815904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-28aec775-d562-4a41-b323-9d166c26a8ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:21.815904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:21.815904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:21.815904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:21.815904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:21.815904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:21.815904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:21.815904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:21.815904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:21.815904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:21.815904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:21.815904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:21.815904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:21.815904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:21.815904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:21.816855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:21.816855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:21.816855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:21.816855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:21.816855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:21.816855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:21.816855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:21.816855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:21.816855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:21.816855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:21.816855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:21.816855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:21.816855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:21.816855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:21.816855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:21.816855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:21.816855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:21.817737 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:21.817737 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:21.817737 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:21.817737 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:21.817737 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 337/691] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:21.824367 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aec775-d562-4a41-b323-9d166c26a8ca None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:21.859427 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:21.863003 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a93d9f-7b34-48fa-84d1-9b6198c0dfdd None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:21.906474 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3486b-5980-432a-a67a-4666afe61b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:21.907907 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3486b-5980-432a-a67a-4666afe61b37 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:21.908354 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3486b-5980-432a-a67a-4666afe61b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:21.908776 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3486b-5980-432a-a67a-4666afe61b37 None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:21.909904 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3486b-5980-432a-a67a-4666afe61b37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:21.910374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3486b-5980-432a-a67a-4666afe61b37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:21.912118 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3486b-5980-432a-a67a-4666afe61b37 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:21.929986 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 355/692] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:21.935898 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3486b-5980-432a-a67a-4666afe61b37 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:21.957025 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aec775-d562-4a41-b323-9d166c26a8ca None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:21.996121 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce0e3e-22c5-46ba-a607-52cfadf1dd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:21.998166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce0e3e-22c5-46ba-a607-52cfadf1dd6c None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:21.998166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce0e3e-22c5-46ba-a607-52cfadf1dd6c None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:21.998166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce0e3e-22c5-46ba-a607-52cfadf1dd6c None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:21.998166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce0e3e-22c5-46ba-a607-52cfadf1dd6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:21.998496 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce0e3e-22c5-46ba-a607-52cfadf1dd6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:22.000148 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce0e3e-22c5-46ba-a607-52cfadf1dd6c None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:22.018430 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 338/693] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:22.021155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce0e3e-22c5-46ba-a607-52cfadf1dd6c None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:22.080562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3486b-5980-432a-a67a-4666afe61b37 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:22.104754 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:22.123009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568971a-7a64-4bb9-8fc5-e8abb249adf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:22.123830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568971a-7a64-4bb9-8fc5-e8abb249adf4 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:22.124121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568971a-7a64-4bb9-8fc5-e8abb249adf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:22.124769 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568971a-7a64-4bb9-8fc5-e8abb249adf4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:22.133097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568971a-7a64-4bb9-8fc5-e8abb249adf4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:22.134577 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568971a-7a64-4bb9-8fc5-e8abb249adf4 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:22.151903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-e568971a-7a64-4bb9-8fc5-e8abb249adf4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 18:27:22.151903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:22.151903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:22.151903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:22.151903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:22.151903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:22.151903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:22.151903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:22.151903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:22.151903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:22.151903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:22.151903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:22.151903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:22.151903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:22.151903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:22.151903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:22.153073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:22.153073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:22.153073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:22.153073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:22.153073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:22.153073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:22.153073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:22.153073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:22.153073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:22.153073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:22.153073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:22.153073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:22.153073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:22.153073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:22.153073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:22.153073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:22.153073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:22.153073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:22.154194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:22.154194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:22.154194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:22.154194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:22.154194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:22.154194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:22.154194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:22.154194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:22.154194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 18:27:22.154194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:22.154194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 356/694] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Jun 27 18:27:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:22.159276 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568971a-7a64-4bb9-8fc5-e8abb249adf4 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:22.223076 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:22.223952 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce0e3e-22c5-46ba-a607-52cfadf1dd6c None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:22.265147 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a83f5-204b-4357-b036-67aec4d0b32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:22.265934 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a83f5-204b-4357-b036-67aec4d0b32f None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:22.266743 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a83f5-204b-4357-b036-67aec4d0b32f None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:22.267028 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a83f5-204b-4357-b036-67aec4d0b32f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:22.274450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a83f5-204b-4357-b036-67aec4d0b32f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:22.275879 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a83f5-204b-4357-b036-67aec4d0b32f None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:22.292730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-a91a83f5-204b-4357-b036-67aec4d0b32f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:22.292730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:22.292730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:22.292730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:22.292730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:22.292730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:22.292730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:22.292730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:22.292730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:22.292730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:22.292730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:22.292730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:22.292730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:22.292730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:22.292730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:22.293643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:22.293643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:22.293643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:22.293643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:22.293643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:22.293643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:22.293643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:22.293643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:22.293643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:22.293643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:22.293643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:22.293643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:22.293643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:22.293643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:22.293643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:22.293643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:22.293643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:22.294532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:22.294532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:22.294532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:22.294532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:22.294532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:22.294532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:22.294532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:22.294532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:22.294532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 339/695] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:22.303489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a83f5-204b-4357-b036-67aec4d0b32f None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:22.378895 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568971a-7a64-4bb9-8fc5-e8abb249adf4 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:22.419451 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29854a5-deae-4a80-82f5-ab75c8c82173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:22.420128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29854a5-deae-4a80-82f5-ab75c8c82173 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:22.420417 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29854a5-deae-4a80-82f5-ab75c8c82173 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:22.420693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29854a5-deae-4a80-82f5-ab75c8c82173 None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:22.421536 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29854a5-deae-4a80-82f5-ab75c8c82173 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:22.421843 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29854a5-deae-4a80-82f5-ab75c8c82173 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:22.423315 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29854a5-deae-4a80-82f5-ab75c8c82173 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:22.448668 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 357/696] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Jun 27 18:27:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:22.455348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29854a5-deae-4a80-82f5-ab75c8c82173 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:22.492867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:22.504655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a83f5-204b-4357-b036-67aec4d0b32f None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:22.560135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e7124-0a11-4849-a00f-79fa646268c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:22.560890 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e7124-0a11-4849-a00f-79fa646268c0 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:22.561242 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e7124-0a11-4849-a00f-79fa646268c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:22.561587 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e7124-0a11-4849-a00f-79fa646268c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:22.562492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e7124-0a11-4849-a00f-79fa646268c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:22.562976 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e7124-0a11-4849-a00f-79fa646268c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:22.564590 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e7124-0a11-4849-a00f-79fa646268c0 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:22.578407 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 340/697] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:22.587460 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e7124-0a11-4849-a00f-79fa646268c0 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:22.595059 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:22.618459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29854a5-deae-4a80-82f5-ab75c8c82173 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:22.659658 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d176d-293b-4636-b2a3-3daccd847193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:22.660568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d176d-293b-4636-b2a3-3daccd847193 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:22.660859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d176d-293b-4636-b2a3-3daccd847193 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:22.661130 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d176d-293b-4636-b2a3-3daccd847193 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:22.707661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e7124-0a11-4849-a00f-79fa646268c0 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:22.746109 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2d176d-293b-4636-b2a3-3daccd847193 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:22.747541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2d176d-293b-4636-b2a3-3daccd847193 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:22.762914 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f871c4d-4f17-4007-81a4-ebc9d82b951d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:22.763990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f871c4d-4f17-4007-81a4-ebc9d82b951d None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:22.764285 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f871c4d-4f17-4007-81a4-ebc9d82b951d None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:22.764610 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f871c4d-4f17-4007-81a4-ebc9d82b951d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:22.765714 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f871c4d-4f17-4007-81a4-ebc9d82b951d None admin] RBAC: Adding query filter params (role.id=c4bc6e72b9d647c5b2485d52d55f7dd4, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=a1529c9a45434a19968e924797688f06) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:22.766072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f871c4d-4f17-4007-81a4-ebc9d82b951d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:22.767701 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f871c4d-4f17-4007-81a4-ebc9d82b951d None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:22.775559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 341/698] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:27:22 2022] GET /identity/v3/role_assignments?user.id=a1529c9a45434a19968e924797688f06&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=c4bc6e72b9d647c5b2485d52d55f7dd4 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:22.777244 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090ba1d-5163-49af-a0c2-7c8add35e8ca None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:22.777532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090ba1d-5163-49af-a0c2-7c8add35e8ca None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:22.777805 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090ba1d-5163-49af-a0c2-7c8add35e8ca None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:22.782561 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 342/699] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:22.785200 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc81c9-f76a-4342-bb63-8d7aa9c61a85 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:22.785481 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc81c9-f76a-4342-bb63-8d7aa9c61a85 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:22.785752 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc81c9-f76a-4342-bb63-8d7aa9c61a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:22.805205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-a1bc81c9-f76a-4342-bb63-8d7aa9c61a85 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:22.806303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-a1bc81c9-f76a-4342-bb63-8d7aa9c61a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:22.851374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:22.857759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:22.875999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 358/700] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:27:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:27:22.882125 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d176d-293b-4636-b2a3-3daccd847193 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:22.985001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 343/701] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:27:23.006971 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d176d-293b-4636-b2a3-3daccd847193 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:23.012570 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc81c9-f76a-4342-bb63-8d7aa9c61a85 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:23.056621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7e74a-a8ee-405a-bb71-aebb45d84d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:23.058060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7e74a-a8ee-405a-bb71-aebb45d84d5e None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:23.058813 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7e74a-a8ee-405a-bb71-aebb45d84d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:23.059144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7e74a-a8ee-405a-bb71-aebb45d84d5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:23.070571 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7e74a-a8ee-405a-bb71-aebb45d84d5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:23.072010 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7e74a-a8ee-405a-bb71-aebb45d84d5e None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:23.093999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-88d7e74a-a8ee-405a-bb71-aebb45d84d5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:23.093999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:23.093999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:23.093999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:23.093999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:23.093999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:23.093999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:23.093999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:23.093999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:23.093999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:23.093999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:23.093999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:23.093999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:23.093999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:23.093999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:23.095171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:23.095171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:23.095171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:23.095171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:23.095171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:23.095171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:23.095171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:23.095171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:23.095171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:23.095171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:23.095171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:23.095171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:23.095171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:23.095171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:23.095171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:23.095171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:23.095171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:23.096502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:23.096502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:23.096502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:23.096502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:23.096502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 359/702] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:23.096850 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b31a7-e0ff-4385-8447-d07e04bceaee None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:23.097084 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b31a7-e0ff-4385-8447-d07e04bceaee None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:23.097356 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b31a7-e0ff-4385-8447-d07e04bceaee None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:23.098859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 360/703] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:23.102475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b31a7-e0ff-4385-8447-d07e04bceaee None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:23.139781 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc81c9-f76a-4342-bb63-8d7aa9c61a85 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:23.142843 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:23.187347 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e26ef6-2e7e-4d86-9c94-47c7f33230df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:23.189216 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e26ef6-2e7e-4d86-9c94-47c7f33230df None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:23.189960 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e26ef6-2e7e-4d86-9c94-47c7f33230df None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:23.190488 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e26ef6-2e7e-4d86-9c94-47c7f33230df None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:23.191598 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e26ef6-2e7e-4d86-9c94-47c7f33230df None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:23.193761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e26ef6-2e7e-4d86-9c94-47c7f33230df None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:23.214145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-00e26ef6-2e7e-4d86-9c94-47c7f33230df None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 18:27:23.214145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:23.214145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:27:23.214145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:27:23.214145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:27:23.214145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:27:23.214145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:27:23.214145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:23.214145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 18:27:23.214145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:23.214145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:23.214145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:23.214145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:23.214145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:23.214145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:23.214145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:23.215217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:23.215217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:23.215217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:23.215217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:23.215217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:23.215217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:23.215217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:23.215217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:27:23.215217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:27:23.215217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:27:23.215217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:27:23.215217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:23.215217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:23.215217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:23.215217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:23.215217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:23.216051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:23.216051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:23.216051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:23.216051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:23.216051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:23.216051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:23.216051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:23.216051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:23.216051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:23.216051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:23.216051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:23.216051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:27:23.216051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:23.216051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 18:27:23.216051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:23.219445 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 344/704] 10.222.0.15 () {56 vars in 1129 bytes} [Mon Jun 27 18:27:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:27:23.221259 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dea1e-b944-4b1e-a4fb-ecf566ff6286 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:23.221698 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dea1e-b944-4b1e-a4fb-ecf566ff6286 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:23.222536 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2dea1e-b944-4b1e-a4fb-ecf566ff6286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:23.239995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b31a7-e0ff-4385-8447-d07e04bceaee None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:23.245885 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-ca2dea1e-b944-4b1e-a4fb-ecf566ff6286 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:23.247615 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-ca2dea1e-b944-4b1e-a4fb-ecf566ff6286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:23.261149 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:23.278671 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafcd5e-f8ff-40f4-aa54-9c62dc986586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:23.279496 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafcd5e-f8ff-40f4-aa54-9c62dc986586 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:23.279879 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafcd5e-f8ff-40f4-aa54-9c62dc986586 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:23.280254 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafcd5e-f8ff-40f4-aa54-9c62dc986586 None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:23.281117 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cafcd5e-f8ff-40f4-aa54-9c62dc986586 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:23.281555 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cafcd5e-f8ff-40f4-aa54-9c62dc986586 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:23.283068 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cafcd5e-f8ff-40f4-aa54-9c62dc986586 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:23.298024 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 361/705] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:23.299055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafcd5e-f8ff-40f4-aa54-9c62dc986586 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:23.344512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 345/706] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:27:23.344512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2dea1e-b944-4b1e-a4fb-ecf566ff6286 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:23.413236 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:23.425006 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafcd5e-f8ff-40f4-aa54-9c62dc986586 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:23.461921 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ec9e6-ab77-412c-a49b-efb464834307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:23.463074 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ec9e6-ab77-412c-a49b-efb464834307 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:23.463632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ec9e6-ab77-412c-a49b-efb464834307 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:23.464378 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ec9e6-ab77-412c-a49b-efb464834307 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:23.465559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ec9e6-ab77-412c-a49b-efb464834307 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:23.466103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ec9e6-ab77-412c-a49b-efb464834307 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:23.467960 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ec9e6-ab77-412c-a49b-efb464834307 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:23.474486 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 362/707] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Jun 27 18:27:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:23.475601 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ec9e6-ab77-412c-a49b-efb464834307 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:23.485062 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2dea1e-b944-4b1e-a4fb-ecf566ff6286 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:23.529682 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb27e4-cd2e-4043-9271-96b0bada587a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:23.530586 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb27e4-cd2e-4043-9271-96b0bada587a None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:23.530973 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb27e4-cd2e-4043-9271-96b0bada587a None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:23.531352 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb27e4-cd2e-4043-9271-96b0bada587a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:23.541410 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cb27e4-cd2e-4043-9271-96b0bada587a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:23.544377 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cb27e4-cd2e-4043-9271-96b0bada587a None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:23.562550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-93cb27e4-cd2e-4043-9271-96b0bada587a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:23.562550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:23.562550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:23.562550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:23.562550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:23.562550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:23.562550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:23.562550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:23.562550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:23.562550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:23.562550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:23.562550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:23.562550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:23.562550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:23.562550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:23.563675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:23.563675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:23.563675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:23.563675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:23.563675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:23.563675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:23.563675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:23.563675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:23.563675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:23.563675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:23.563675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:23.563675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:23.563675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:23.563675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:23.563675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:23.563675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:23.563675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:23.564642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:23.564642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:23.564642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:23.564642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:23.564642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:23.564642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:23.564642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:23.564642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:23.565503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 346/708] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:23.566267 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb27e4-cd2e-4043-9271-96b0bada587a None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:23.571983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:23.587611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ec9e6-ab77-412c-a49b-efb464834307 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:23.634694 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2c6b0-dfba-4f28-990a-2154d79080b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:23.636010 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2c6b0-dfba-4f28-990a-2154d79080b5 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:23.636298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2c6b0-dfba-4f28-990a-2154d79080b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:23.636574 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2c6b0-dfba-4f28-990a-2154d79080b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:23.652387 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2c6b0-dfba-4f28-990a-2154d79080b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:23.654136 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2c6b0-dfba-4f28-990a-2154d79080b5 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:23.677665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-f4e2c6b0-dfba-4f28-990a-2154d79080b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:23.677665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:23.677665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:23.677665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:23.677665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:23.677665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:23.677665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:23.677665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:23.677665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:23.677665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:23.677665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:23.677665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:23.677665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:23.677665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:23.677665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:23.679054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:23.679054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:23.679054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:23.679054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:23.679054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:23.679054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:23.679054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:23.679054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:23.679054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:23.679054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:23.679054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:23.679054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:23.679054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:23.679054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:23.679054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:23.679054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:23.679054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:23.680469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:23.680469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:23.680469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:23.680469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:23.682790 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 363/709] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:23.684065 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2c6b0-dfba-4f28-990a-2154d79080b5 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:23.695452 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb27e4-cd2e-4043-9271-96b0bada587a None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:23.702590 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:23.735208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198aef17-4f75-4306-8823-8ea0ab7628c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:23.736350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198aef17-4f75-4306-8823-8ea0ab7628c9 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:23.736836 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198aef17-4f75-4306-8823-8ea0ab7628c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:23.737387 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198aef17-4f75-4306-8823-8ea0ab7628c9 None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:23.738351 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198aef17-4f75-4306-8823-8ea0ab7628c9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:23.738898 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198aef17-4f75-4306-8823-8ea0ab7628c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:23.740636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198aef17-4f75-4306-8823-8ea0ab7628c9 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:23.753957 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 347/710] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Jun 27 18:27:23 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:23.755054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198aef17-4f75-4306-8823-8ea0ab7628c9 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:23.871106 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2c6b0-dfba-4f28-990a-2154d79080b5 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:23.920399 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:23.940944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4df29c-fd26-426b-97da-fc87f2339481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:23.942133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4df29c-fd26-426b-97da-fc87f2339481 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:23.942729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4df29c-fd26-426b-97da-fc87f2339481 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:23.943211 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4df29c-fd26-426b-97da-fc87f2339481 None admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:23.944258 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4df29c-fd26-426b-97da-fc87f2339481 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:23.944573 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4df29c-fd26-426b-97da-fc87f2339481 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:23.946016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4df29c-fd26-426b-97da-fc87f2339481 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:23.960347 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 364/711] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:23.967290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4df29c-fd26-426b-97da-fc87f2339481 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:23.976226 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198aef17-4f75-4306-8823-8ea0ab7628c9 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:24.030996 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bfc58-11b4-443a-96b8-83e20b670fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:24.031664 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bfc58-11b4-443a-96b8-83e20b670fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:24.031948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bfc58-11b4-443a-96b8-83e20b670fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:24.032226 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bfc58-11b4-443a-96b8-83e20b670fb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:24.032956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bfc58-11b4-443a-96b8-83e20b670fb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:24.033265 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bfc58-11b4-443a-96b8-83e20b670fb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:24.035917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bfc58-11b4-443a-96b8-83e20b670fb3 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:24.062972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 348/712] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:24.065643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bfc58-11b4-443a-96b8-83e20b670fb3 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:24.079142 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:24.127428 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4df29c-fd26-426b-97da-fc87f2339481 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:24.164862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe99ea3-78e7-4c5a-a462-6aaaa923d64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:24.165838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe99ea3-78e7-4c5a-a462-6aaaa923d64f None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:24.166335 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe99ea3-78e7-4c5a-a462-6aaaa923d64f None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:24.167533 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe99ea3-78e7-4c5a-a462-6aaaa923d64f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:24.168630 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe99ea3-78e7-4c5a-a462-6aaaa923d64f None admin] RBAC: Adding query filter params (role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=1dddd4efb44c4152bb4151f7715f2223) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:24.169179 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe99ea3-78e7-4c5a-a462-6aaaa923d64f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:24.170943 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe99ea3-78e7-4c5a-a462-6aaaa923d64f None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:24.177327 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 365/713] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:27:23 2022] GET /identity/v3/role_assignments?user.id=1dddd4efb44c4152bb4151f7715f2223&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:24.178934 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e29b74-f5ba-434f-bd3b-f016d22d405b None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:24.179472 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e29b74-f5ba-434f-bd3b-f016d22d405b None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:24.179945 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e29b74-f5ba-434f-bd3b-f016d22d405b None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:24.181790 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 366/714] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:24.193710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c511e-b175-4d2c-856c-471c5a0faaf9 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:24.194637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c511e-b175-4d2c-856c-471c5a0faaf9 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:24.198938 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c511e-b175-4d2c-856c-471c5a0faaf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:24.228396 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bfc58-11b4-443a-96b8-83e20b670fb3 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:24.230750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-bd0c511e-b175-4d2c-856c-471c5a0faaf9 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:24.234055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-bd0c511e-b175-4d2c-856c-471c5a0faaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:24.290741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:24.308784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61742d-6c32-472a-8714-ac56e57ad9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:24.309470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61742d-6c32-472a-8714-ac56e57ad9ce None admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:24.309748 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61742d-6c32-472a-8714-ac56e57ad9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:24.310051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61742d-6c32-472a-8714-ac56e57ad9ce None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:24.310797 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61742d-6c32-472a-8714-ac56e57ad9ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:24.312567 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61742d-6c32-472a-8714-ac56e57ad9ce None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:24.359267 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 367/715] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:27:24.381986 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c511e-b175-4d2c-856c-471c5a0faaf9 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:24.427944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 349/716] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Jun 27 18:27:24 2022] POST /identity/v3/users => generated 254 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:27:24.451208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:24.497589 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c511e-b175-4d2c-856c-471c5a0faaf9 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:24.549943 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aaf2a8-db31-46f8-a144-0b52bd8be2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:24.550786 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aaf2a8-db31-46f8-a144-0b52bd8be2ef None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:24.551068 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aaf2a8-db31-46f8-a144-0b52bd8be2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:24.551343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aaf2a8-db31-46f8-a144-0b52bd8be2ef None admin] PATH_INFO: `/v3/services/network` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:24.552006 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aaf2a8-db31-46f8-a144-0b52bd8be2ef None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:24.553544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aaf2a8-db31-46f8-a144-0b52bd8be2ef None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:24.573600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-96aaf2a8-db31-46f8-a144-0b52bd8be2ef None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 18:27:24.573600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:24.573600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:27:24.573600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:27:24.573600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:27:24.573600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:27:24.573600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:27:24.573600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:24.573600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 18:27:24.573600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:24.573600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:24.573600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:24.573600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:24.573600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:24.573600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:24.573600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:24.573600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:24.574767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:24.574767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:24.574767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:24.574767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:24.574767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:24.574767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:24.574767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:27:24.574767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:27:24.574767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:27:24.574767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:27:24.574767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:24.574767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:24.574767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:24.574767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:24.574767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:24.574767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:24.575609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:24.575609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:24.575609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:24.575609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:24.575609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:24.575609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:24.575609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:24.575609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:24.575609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:24.575609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:24.575609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:27:24.575609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:24.575609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 18:27:24.575609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:24.575609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 368/717] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:24 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:27:24.580899 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61742d-6c32-472a-8714-ac56e57ad9ce None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:24.701301 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61742d-6c32-472a-8714-ac56e57ad9ce None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:24.741033 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356260b3-a3a8-41ee-9d70-12daaf8e3905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:24.741707 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356260b3-a3a8-41ee-9d70-12daaf8e3905 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:24.741989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356260b3-a3a8-41ee-9d70-12daaf8e3905 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:24.742262 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356260b3-a3a8-41ee-9d70-12daaf8e3905 None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:24.743010 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356260b3-a3a8-41ee-9d70-12daaf8e3905 None admin] RBAC: Adding query filter params (name=network) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:24.743316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356260b3-a3a8-41ee-9d70-12daaf8e3905 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:24.744694 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356260b3-a3a8-41ee-9d70-12daaf8e3905 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:24.753156 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 350/718] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Jun 27 18:27:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:24.758777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aaf2a8-db31-46f8-a144-0b52bd8be2ef None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:24.875190 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aaf2a8-db31-46f8-a144-0b52bd8be2ef None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:24.905816 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:24.916723 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01bb5bb-2867-4b88-9588-576b541d048e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:24.917675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bb5bb-2867-4b88-9588-576b541d048e None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:24.918055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bb5bb-2867-4b88-9588-576b541d048e None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:24.918452 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bb5bb-2867-4b88-9588-576b541d048e None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:24.919288 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01bb5bb-2867-4b88-9588-576b541d048e None admin] RBAC: Adding query filter params (type=network) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:24.919694 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01bb5bb-2867-4b88-9588-576b541d048e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:24.921168 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01bb5bb-2867-4b88-9588-576b541d048e None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:24.930989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 369/719] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Jun 27 18:27:24 2022] GET /identity/v3/services?type=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:25.190282 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28c4c6-bb52-414a-9f2f-c875e78b3085 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:25.190704 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28c4c6-bb52-414a-9f2f-c875e78b3085 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:25.190918 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28c4c6-bb52-414a-9f2f-c875e78b3085 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:25.192633 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 351/720] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:25.198663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d600431-3373-4cd5-8876-a451a7713785 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:25.198957 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d600431-3373-4cd5-8876-a451a7713785 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:25.199233 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d600431-3373-4cd5-8876-a451a7713785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:25.221177 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-4d600431-3373-4cd5-8876-a451a7713785 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:25.222403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-4d600431-3373-4cd5-8876-a451a7713785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:25.306980 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 370/721] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:27:25.330356 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28c4c6-bb52-414a-9f2f-c875e78b3085 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:25.406994 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:25.468938 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28c4c6-bb52-414a-9f2f-c875e78b3085 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:25.522551 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb8240-3a44-4e0e-b410-0833cf25228c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:25.523339 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb8240-3a44-4e0e-b410-0833cf25228c None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:25.523672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb8240-3a44-4e0e-b410-0833cf25228c None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:25.523944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb8240-3a44-4e0e-b410-0833cf25228c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:25.536177 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb8240-3a44-4e0e-b410-0833cf25228c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:25.544719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-2dbb8240-3a44-4e0e-b410-0833cf25228c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:27:25.544719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:25.544719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:25.544719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:25.544719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:25.544719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:25.544719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:25.544719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:25.544719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:25.544719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:25.544719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:25.544719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:25.544719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:25.544719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:25.544719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:25.544719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:25.545675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:25.545675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:25.545675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:25.545675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:25.545675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:25.545675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:25.545675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:25.545675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:25.545675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:25.545675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:25.545675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:25.545675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:25.545675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:25.545675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:25.545675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:25.545675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:25.546559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:25.546559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:25.546559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:25.546559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:25.546559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:25.546559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:25.546559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:25.546559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:25.546559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:25.546559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:25.546559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:27:25.546559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:25.546559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 352/722] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:27:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:25.555475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d600431-3373-4cd5-8876-a451a7713785 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:25.696420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d600431-3373-4cd5-8876-a451a7713785 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:25.722907 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:25.747725 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b2cc0-ae49-4016-b8e7-ab86eb2769dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:25.748622 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b2cc0-ae49-4016-b8e7-ab86eb2769dd None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:25.748819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b2cc0-ae49-4016-b8e7-ab86eb2769dd None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:25.748971 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b2cc0-ae49-4016-b8e7-ab86eb2769dd None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:25.749689 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b2cc0-ae49-4016-b8e7-ab86eb2769dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:25.749999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b2cc0-ae49-4016-b8e7-ab86eb2769dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:25.751426 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b2cc0-ae49-4016-b8e7-ab86eb2769dd None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:25.766545 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 371/723] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:27:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:25.771736 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb8240-3a44-4e0e-b410-0833cf25228c None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:25.928080 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb8240-3a44-4e0e-b410-0833cf25228c None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:25.939252 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:25.978987 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0ca6e-0a06-453c-889d-6b9e11866c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:25.980509 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0ca6e-0a06-453c-889d-6b9e11866c5b None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:25.980859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0ca6e-0a06-453c-889d-6b9e11866c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:25.981140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0ca6e-0a06-453c-889d-6b9e11866c5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:26.002509 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0ca6e-0a06-453c-889d-6b9e11866c5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:26.004017 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0ca6e-0a06-453c-889d-6b9e11866c5b None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:26.025533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-dea0ca6e-0a06-453c-889d-6b9e11866c5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:26.025533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:26.025533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:26.025533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:26.025533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:26.025533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:26.025533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:26.025533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:26.025533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:26.025533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:26.025533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:26.025533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:26.025533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:26.025533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:26.025533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:26.028142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:26.028142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:26.028142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:26.028142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:26.028142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:26.028142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:26.028142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:26.028142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:26.028142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:26.028142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:26.028142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:26.028142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:26.028142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:26.028142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:26.028142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:26.028142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:26.028142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:26.029389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:26.029389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:26.029389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:26.029389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:26.029389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 353/724] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:26.033050 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b2cc0-ae49-4016-b8e7-ab86eb2769dd None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:26.186921 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b2cc0-ae49-4016-b8e7-ab86eb2769dd None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:26.235645 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412d350-840b-449b-8680-9ea94971a476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:26.236645 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412d350-840b-449b-8680-9ea94971a476 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:26.237145 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412d350-840b-449b-8680-9ea94971a476 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:26.237640 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412d350-840b-449b-8680-9ea94971a476 None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:26.238591 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f412d350-840b-449b-8680-9ea94971a476 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:26.239118 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f412d350-840b-449b-8680-9ea94971a476 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:26.240695 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f412d350-840b-449b-8680-9ea94971a476 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:26.257266 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 372/725] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:26.263238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0ca6e-0a06-453c-889d-6b9e11866c5b None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:26.301171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641d054-eb02-4c60-8f4b-15c5e93228d4 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:26.301508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641d054-eb02-4c60-8f4b-15c5e93228d4 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:26.301739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2641d054-eb02-4c60-8f4b-15c5e93228d4 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:26.303451 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 373/726] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:27:26.308371 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b32b7-8af0-46f2-ba17-c5b399da5daa None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:26.308664 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b32b7-8af0-46f2-ba17-c5b399da5daa None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:26.308946 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b32b7-8af0-46f2-ba17-c5b399da5daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:26.327310 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-d08b32b7-8af0-46f2-ba17-c5b399da5daa None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:26.328569 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-d08b32b7-8af0-46f2-ba17-c5b399da5daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:26.366289 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:26.402351 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 374/727] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 18:27:26.425661 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b32b7-8af0-46f2-ba17-c5b399da5daa None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:26.488670 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0ca6e-0a06-453c-889d-6b9e11866c5b None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:26.536879 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b32b7-8af0-46f2-ba17-c5b399da5daa None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:26.571668 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824e04b-75ab-4fcb-8fea-757bf583fa38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:26.572476 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824e04b-75ab-4fcb-8fea-757bf583fa38 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:26.572809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824e04b-75ab-4fcb-8fea-757bf583fa38 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:26.573289 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824e04b-75ab-4fcb-8fea-757bf583fa38 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:26.575251 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca8a53-4539-4887-87e3-673dd811c835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:26.575971 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca8a53-4539-4887-87e3-673dd811c835 None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:26.576258 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca8a53-4539-4887-87e3-673dd811c835 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:26.576535 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca8a53-4539-4887-87e3-673dd811c835 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:26.577261 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbca8a53-4539-4887-87e3-673dd811c835 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:26.578670 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbca8a53-4539-4887-87e3-673dd811c835 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:26.590021 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824e04b-75ab-4fcb-8fea-757bf583fa38 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:26.591768 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824e04b-75ab-4fcb-8fea-757bf583fa38 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:26.607983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-c824e04b-75ab-4fcb-8fea-757bf583fa38 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 18:27:26.607983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:26.607983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:26.607983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:26.607983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:26.607983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:26.607983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:26.607983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:26.607983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:26.607983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:26.607983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:26.607983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:26.607983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:26.607983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:26.607983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:26.607983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:26.608920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:26.608920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:26.608920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:26.608920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:26.608920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:26.608920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:26.608920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:26.608920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:26.608920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:26.608920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:26.608920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:26.608920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:26.608920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:26.608920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:26.608920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:26.608920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:26.608920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:26.608920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:26.609863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:26.609863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:26.609863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:26.609863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:26.609863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:26.609863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:26.609863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:26.609863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:26.609863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 18:27:26.609863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:26.609863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 354/728] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:27:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:26.614877 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824e04b-75ab-4fcb-8fea-757bf583fa38 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:26.645217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:26.694651 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 375/729] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Jun 27 18:27:26 2022] POST /identity/v3/services => generated 251 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:27:26.726019 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824e04b-75ab-4fcb-8fea-757bf583fa38 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:26.819975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63755bd7-b0a9-44d6-ac97-6edbf6b5b3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:26.820685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63755bd7-b0a9-44d6-ac97-6edbf6b5b3a3 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:26.820974 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63755bd7-b0a9-44d6-ac97-6edbf6b5b3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:26.821253 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63755bd7-b0a9-44d6-ac97-6edbf6b5b3a3 None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:26.822113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63755bd7-b0a9-44d6-ac97-6edbf6b5b3a3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:26.822462 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63755bd7-b0a9-44d6-ac97-6edbf6b5b3a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:26.824995 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63755bd7-b0a9-44d6-ac97-6edbf6b5b3a3 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:26.855549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 355/730] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Jun 27 18:27:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:26.861604 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca8a53-4539-4887-87e3-673dd811c835 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:26.903518 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:26.954483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a969315-bc71-4672-8200-0698879bb561 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:26.955121 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a969315-bc71-4672-8200-0698879bb561 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:26.955607 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a969315-bc71-4672-8200-0698879bb561 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:26.957436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 356/731] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:26.963272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99577b49-5006-4e11-b66d-6eccf79797de None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:26.964343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99577b49-5006-4e11-b66d-6eccf79797de None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:26.964920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99577b49-5006-4e11-b66d-6eccf79797de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:26.979011 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca8a53-4539-4887-87e3-673dd811c835 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:26.985227 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-99577b49-5006-4e11-b66d-6eccf79797de None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:26.986830 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-99577b49-5006-4e11-b66d-6eccf79797de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:27.022979 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e202a0d-caf3-4e84-af98-0c70ac73374d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:27.023556 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e202a0d-caf3-4e84-af98-0c70ac73374d None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:27.023842 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e202a0d-caf3-4e84-af98-0c70ac73374d None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:27.024119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e202a0d-caf3-4e84-af98-0c70ac73374d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:27.074523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 357/732] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:26 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:27:27.103279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e202a0d-caf3-4e84-af98-0c70ac73374d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:27.105126 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e202a0d-caf3-4e84-af98-0c70ac73374d None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:27.106991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99577b49-5006-4e11-b66d-6eccf79797de None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:27.117265 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:27.144309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:27.221341 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99577b49-5006-4e11-b66d-6eccf79797de None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:27.242478 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 376/733] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:27:26 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 18:27:27.244102 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a111b3-44a0-4045-8aaf-7061e592ae2b None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:27.244463 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a111b3-44a0-4045-8aaf-7061e592ae2b None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:27.244993 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a111b3-44a0-4045-8aaf-7061e592ae2b None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:27.246722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 377/734] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:27.251235 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a111b3-44a0-4045-8aaf-7061e592ae2b None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:27.263168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcfd3ea-765c-48da-8a01-ca00743ce6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:27.263965 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfd3ea-765c-48da-8a01-ca00743ce6ec None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:27.264251 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfd3ea-765c-48da-8a01-ca00743ce6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:27.264874 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfd3ea-765c-48da-8a01-ca00743ce6ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:27.279332 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcfd3ea-765c-48da-8a01-ca00743ce6ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:27.289949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-2fcfd3ea-765c-48da-8a01-ca00743ce6ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:27:27.289949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:27.289949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:27.289949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:27.289949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:27.289949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:27.289949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:27.289949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:27.289949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:27.289949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:27.289949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:27.289949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:27.289949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:27.289949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:27.289949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:27.289949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:27.291141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:27.291141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:27.291141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:27.291141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:27.291141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:27.291141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:27.291141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:27.291141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:27.291141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:27.291141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:27.291141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:27.291141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:27.291141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:27.291141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:27.291141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:27.291141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:27.292224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:27.292224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:27.292224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:27.292224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:27.292224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:27.292224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:27.292224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:27.292224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:27.292224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:27.292224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:27.292224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:27:27.292224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:27.296261 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 358/735] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:27:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:27:27.298085 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ef7dc-e627-4156-8e4a-fea559b890f7 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:27.298374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ef7dc-e627-4156-8e4a-fea559b890f7 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:27.299046 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ef7dc-e627-4156-8e4a-fea559b890f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:27.318344 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-520ef7dc-e627-4156-8e4a-fea559b890f7 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:27.319723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-520ef7dc-e627-4156-8e4a-fea559b890f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:27.427440 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a111b3-44a0-4045-8aaf-7061e592ae2b None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:27.465571 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:27.467863 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:27.475211 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43a983-cedb-4100-8bec-c77c50bdab60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:27.482421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 359/736] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:27:27.482421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ef7dc-e627-4156-8e4a-fea559b890f7 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:27.482982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43a983-cedb-4100-8bec-c77c50bdab60 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:27.483089 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43a983-cedb-4100-8bec-c77c50bdab60 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:27.483344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43a983-cedb-4100-8bec-c77c50bdab60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:27.505536 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43a983-cedb-4100-8bec-c77c50bdab60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:27.507054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43a983-cedb-4100-8bec-c77c50bdab60 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:27.529413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-6c43a983-cedb-4100-8bec-c77c50bdab60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:27.529413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:27.529413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:27.529413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:27.529413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:27.529413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:27.529413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:27.529413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:27.529413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:27.529413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:27.529413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:27.529413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:27.529413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:27.529413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:27.529413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:27.530338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:27.530338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:27.530338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:27.530338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:27.530338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:27.530338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:27.530338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:27.530338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:27.530338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:27.530338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:27.530338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:27.530338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:27.530338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:27.530338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:27.530338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:27.530338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:27.530338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:27.531664 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:27.531664 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:27.531664 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:27.531664 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:27.531664 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 378/737] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:27.531954 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43a983-cedb-4100-8bec-c77c50bdab60 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:27.602623 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ef7dc-e627-4156-8e4a-fea559b890f7 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:27.638002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd99a7b-aac0-4a69-af61-ad043f27dc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:27.638699 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd99a7b-aac0-4a69-af61-ad043f27dc71 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:27.638984 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd99a7b-aac0-4a69-af61-ad043f27dc71 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:27.639329 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd99a7b-aac0-4a69-af61-ad043f27dc71 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:27.640073 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd99a7b-aac0-4a69-af61-ad043f27dc71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:27.640377 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd99a7b-aac0-4a69-af61-ad043f27dc71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:27.641736 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd99a7b-aac0-4a69-af61-ad043f27dc71 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:27.642595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43a983-cedb-4100-8bec-c77c50bdab60 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:27.649939 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 360/738] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:27:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:27.650823 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd99a7b-aac0-4a69-af61-ad043f27dc71 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:27.684197 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e9704-012b-4f1f-ae79-70d693c84459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:27.685319 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e9704-012b-4f1f-ae79-70d693c84459 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:27.685872 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e9704-012b-4f1f-ae79-70d693c84459 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:27.686506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e9704-012b-4f1f-ae79-70d693c84459 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:27.700572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e9704-012b-4f1f-ae79-70d693c84459 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:27.708398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-121e9704-012b-4f1f-ae79-70d693c84459 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:27.708398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:27.708398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:27.708398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:27.708398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:27.708398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:27.708398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:27.708398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:27.708398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:27.708398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:27.708398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:27.708398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:27.708398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:27.708398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:27.708398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:27.708398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:27.709527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:27.709527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:27.709527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:27.709527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:27.709527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:27.709527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:27.709527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:27.709527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:27.709527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:27.709527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:27.709527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:27.709527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:27.709527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:27.709527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:27.709527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:27.709527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:27.710427 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:27.710427 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:27.710427 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:27.710427 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:27.710427 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:27.710427 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:27.710427 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:27.710427 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:27.710427 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:27.710427 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:27.710427 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:27.710427 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:27.711088 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:27.711162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 379/739] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:27:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:27.711162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0480aff-77ad-41c8-a8ec-1b203312a219 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:27.711162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0480aff-77ad-41c8-a8ec-1b203312a219 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:27.711355 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0480aff-77ad-41c8-a8ec-1b203312a219 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:27.713033 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 380/740] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:27.714055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0480aff-77ad-41c8-a8ec-1b203312a219 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:27.762265 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:27.769126 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd99a7b-aac0-4a69-af61-ad043f27dc71 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:27.806010 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efe807-7afa-4616-824e-7c95d1a22a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:27.806619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efe807-7afa-4616-824e-7c95d1a22a3e None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:27.806906 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efe807-7afa-4616-824e-7c95d1a22a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:27.807228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efe807-7afa-4616-824e-7c95d1a22a3e None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:27.807946 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efe807-7afa-4616-824e-7c95d1a22a3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:27.808250 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efe807-7afa-4616-824e-7c95d1a22a3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:27.809620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efe807-7afa-4616-824e-7c95d1a22a3e None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:27.821936 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0480aff-77ad-41c8-a8ec-1b203312a219 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:27.827254 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 361/741] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:27.827254 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50410a-2062-45c2-b004-82762a9cc4b3 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:27.827254 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50410a-2062-45c2-b004-82762a9cc4b3 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:27.827254 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50410a-2062-45c2-b004-82762a9cc4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:27.843360 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-bd50410a-2062-45c2-b004-82762a9cc4b3 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:27.844566 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-bd50410a-2062-45c2-b004-82762a9cc4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:27.859252 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5727f8-2cdd-4514-b673-5db0d38aab89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:27.860079 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5727f8-2cdd-4514-b673-5db0d38aab89 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:27.860377 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5727f8-2cdd-4514-b673-5db0d38aab89 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:27.860641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5727f8-2cdd-4514-b673-5db0d38aab89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:27.872458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5727f8-2cdd-4514-b673-5db0d38aab89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:27.874445 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5727f8-2cdd-4514-b673-5db0d38aab89 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:27.894009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-5e5727f8-2cdd-4514-b673-5db0d38aab89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:27.894009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:27.894009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:27.894009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:27.894009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:27.894009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:27.894009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:27.894009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:27.894009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:27.894009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:27.894009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:27.894009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:27.894009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:27.894009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:27.894009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:27.894994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:27.894994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:27.894994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:27.894994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:27.894994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:27.894994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:27.894994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:27.894994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:27.894994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:27.894994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:27.894994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:27.894994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:27.894994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:27.894994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:27.894994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:27.894994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:27.894994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:27.895880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:27.895880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:27.895880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:27.895880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:27.895880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 381/742] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:27.896168 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5727f8-2cdd-4514-b673-5db0d38aab89 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:27.930403 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 362/743] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 18:27:27.930403 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50410a-2062-45c2-b004-82762a9cc4b3 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:27.984420 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:27.994607 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:28.002100 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5727f8-2cdd-4514-b673-5db0d38aab89 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:28.037159 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e01df6-7027-4aa0-833c-5ce95e0c76bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:28.037822 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e01df6-7027-4aa0-833c-5ce95e0c76bb None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:28.038111 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e01df6-7027-4aa0-833c-5ce95e0c76bb None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:28.038475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e01df6-7027-4aa0-833c-5ce95e0c76bb None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:28.038809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50410a-2062-45c2-b004-82762a9cc4b3 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:28.039221 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e01df6-7027-4aa0-833c-5ce95e0c76bb None admin] RBAC: Adding query filter params (name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:28.039922 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e01df6-7027-4aa0-833c-5ce95e0c76bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:28.043424 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e01df6-7027-4aa0-833c-5ce95e0c76bb None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:28.053521 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 382/744] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Jun 27 18:27:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:28.054329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e01df6-7027-4aa0-833c-5ce95e0c76bb None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:28.078467 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49a973-bc60-4251-a194-2c0b03eda002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:28.079587 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49a973-bc60-4251-a194-2c0b03eda002 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:28.080063 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49a973-bc60-4251-a194-2c0b03eda002 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:28.080674 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49a973-bc60-4251-a194-2c0b03eda002 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:28.089230 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49a973-bc60-4251-a194-2c0b03eda002 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:28.091236 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49a973-bc60-4251-a194-2c0b03eda002 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:28.109474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-ca49a973-bc60-4251-a194-2c0b03eda002 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:28.109474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:28.109474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:28.109474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:28.109474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:28.109474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:28.109474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:28.109474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:28.109474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:28.109474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:28.109474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:28.109474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:28.109474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:28.109474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:28.109474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:28.110435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:28.110435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:28.110435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:28.110435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:28.110435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:28.110435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:28.110435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:28.110435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:28.110435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:28.110435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:28.110435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:28.110435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:28.110435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:28.110435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:28.110435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:28.110435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:28.110435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:28.111469 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:28.111469 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:28.111469 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:28.111469 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:28.111469 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:28.111469 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:28.111469 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:28.111469 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:28.113286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 363/745] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:28.114178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49a973-bc60-4251-a194-2c0b03eda002 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:28.168110 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e01df6-7027-4aa0-833c-5ce95e0c76bb None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:28.203688 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e7f8f-3588-476c-800d-0f57f6b40e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:28.204355 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e7f8f-3588-476c-800d-0f57f6b40e3e None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:28.204628 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e7f8f-3588-476c-800d-0f57f6b40e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:28.204907 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e7f8f-3588-476c-800d-0f57f6b40e3e None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:28.205630 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e7f8f-3588-476c-800d-0f57f6b40e3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:28.205939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e7f8f-3588-476c-800d-0f57f6b40e3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:28.207348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e7f8f-3588-476c-800d-0f57f6b40e3e None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:28.220018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 383/746] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:28.220810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e7f8f-3588-476c-800d-0f57f6b40e3e None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:28.221035 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49a973-bc60-4251-a194-2c0b03eda002 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:28.232214 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:28.243017 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:28.262816 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e54322-10bd-41ad-af02-bc80809f749b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:28.263833 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e54322-10bd-41ad-af02-bc80809f749b None admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:28.264116 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e54322-10bd-41ad-af02-bc80809f749b None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:28.264383 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e54322-10bd-41ad-af02-bc80809f749b None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:28.265091 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e54322-10bd-41ad-af02-bc80809f749b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:28.266464 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e54322-10bd-41ad-af02-bc80809f749b None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:28.326715 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e7f8f-3588-476c-800d-0f57f6b40e3e None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:28.348222 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 364/747] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Jun 27 18:27:28 2022] POST /identity/v3/services => generated 240 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:27:28.349209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e54322-10bd-41ad-af02-bc80809f749b None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:28.382848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57025bc7-b94f-4718-9602-f495fcd112ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:28.384057 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57025bc7-b94f-4718-9602-f495fcd112ca None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:28.385232 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57025bc7-b94f-4718-9602-f495fcd112ca None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:28.386323 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57025bc7-b94f-4718-9602-f495fcd112ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:28.402065 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57025bc7-b94f-4718-9602-f495fcd112ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:28.405461 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57025bc7-b94f-4718-9602-f495fcd112ca None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:28.432000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-57025bc7-b94f-4718-9602-f495fcd112ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:28.432000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:28.432000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:28.432000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:28.432000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:28.432000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:28.432000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:28.432000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:28.432000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:28.432000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:28.432000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:28.432000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:28.432000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:28.432000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:28.432000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:28.433503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:28.433503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:28.433503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:28.433503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:28.433503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:28.433503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:28.433503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:28.433503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:28.433503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:28.433503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:28.433503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:28.433503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:28.433503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:28.433503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:28.433503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:28.433503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:28.433503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:28.434411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:28.434411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:28.434411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:28.434411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:28.444901 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 384/748] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:28.444901 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57025bc7-b94f-4718-9602-f495fcd112ca None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:28.504920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e54322-10bd-41ad-af02-bc80809f749b None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:28.531666 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:28.545763 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969e75d-11e6-4823-9bb2-971e694f426a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:28.546806 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969e75d-11e6-4823-9bb2-971e694f426a None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:28.547278 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969e75d-11e6-4823-9bb2-971e694f426a None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:28.547774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969e75d-11e6-4823-9bb2-971e694f426a None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:28.548751 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969e75d-11e6-4823-9bb2-971e694f426a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:28.549258 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969e75d-11e6-4823-9bb2-971e694f426a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:28.550929 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969e75d-11e6-4823-9bb2-971e694f426a None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:28.568779 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 365/749] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:28.570170 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969e75d-11e6-4823-9bb2-971e694f426a None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:28.591191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57025bc7-b94f-4718-9602-f495fcd112ca None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:28.605853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:28.632848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62ad7e-6839-43c7-bcd2-365f75270cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:28.633642 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62ad7e-6839-43c7-bcd2-365f75270cbb None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:28.633925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62ad7e-6839-43c7-bcd2-365f75270cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:28.634201 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62ad7e-6839-43c7-bcd2-365f75270cbb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:28.642848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62ad7e-6839-43c7-bcd2-365f75270cbb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:28.644318 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62ad7e-6839-43c7-bcd2-365f75270cbb None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:28.659982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-bf62ad7e-6839-43c7-bcd2-365f75270cbb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 18:27:28.659982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:28.659982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:28.659982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:28.659982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:28.659982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:28.659982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:28.659982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:28.659982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:28.659982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:28.659982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:28.659982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:28.659982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:28.659982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:28.659982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:28.659982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:28.661002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:28.661002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:28.661002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:28.661002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:28.661002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:28.661002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:28.661002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:28.661002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:28.661002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:28.661002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:28.661002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:28.661002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:28.661002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:28.661002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:28.661002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:28.661002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:28.661002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:28.661002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:28.661939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:28.661939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:28.661939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:28.661939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:28.661939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:28.661939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:28.661939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:28.661939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:28.661939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 18:27:28.661939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:28.661939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 385/750] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Jun 27 18:27:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:28.668580 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62ad7e-6839-43c7-bcd2-365f75270cbb None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:28.694532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969e75d-11e6-4823-9bb2-971e694f426a None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:28.748144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bcf6fb-19a1-43e9-9019-aacb3e848157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:28.748815 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bcf6fb-19a1-43e9-9019-aacb3e848157 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:28.749101 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bcf6fb-19a1-43e9-9019-aacb3e848157 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:28.749376 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bcf6fb-19a1-43e9-9019-aacb3e848157 None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:28.750107 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bcf6fb-19a1-43e9-9019-aacb3e848157 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:28.750446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bcf6fb-19a1-43e9-9019-aacb3e848157 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:28.751846 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bcf6fb-19a1-43e9-9019-aacb3e848157 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:28.769388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 366/751] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:28.769388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bcf6fb-19a1-43e9-9019-aacb3e848157 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:28.808007 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62ad7e-6839-43c7-bcd2-365f75270cbb None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:28.837730 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:28.854706 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e8bab-4a58-4f5e-8d1e-a01aea709d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:28.855749 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e8bab-4a58-4f5e-8d1e-a01aea709d49 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:28.856094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e8bab-4a58-4f5e-8d1e-a01aea709d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:28.856320 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e8bab-4a58-4f5e-8d1e-a01aea709d49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:28.857429 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e8bab-4a58-4f5e-8d1e-a01aea709d49 None admin] RBAC: Adding query filter params (role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=a1529c9a45434a19968e924797688f06) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:28.857745 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e8bab-4a58-4f5e-8d1e-a01aea709d49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:28.860209 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e8bab-4a58-4f5e-8d1e-a01aea709d49 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:28.867358 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 386/752] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:27:28 2022] GET /identity/v3/role_assignments?user.id=a1529c9a45434a19968e924797688f06&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:28.867658 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e8bab-4a58-4f5e-8d1e-a01aea709d49 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:28.887922 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:28.920791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bcf6fb-19a1-43e9-9019-aacb3e848157 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:28.988144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e8bab-4a58-4f5e-8d1e-a01aea709d49 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:28.990354 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaaaba5-1def-4311-b7ba-88364c24bb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:28.991055 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaaaba5-1def-4311-b7ba-88364c24bb42 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:28.991342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaaaba5-1def-4311-b7ba-88364c24bb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:28.991624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaaaba5-1def-4311-b7ba-88364c24bb42 None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:28.992485 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaaaba5-1def-4311-b7ba-88364c24bb42 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:28.992799 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaaaba5-1def-4311-b7ba-88364c24bb42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:28.994272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaaaba5-1def-4311-b7ba-88364c24bb42 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:29.036951 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 367/753] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Jun 27 18:27:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:29.043229 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac57322-73bf-4536-9ccb-41409d604286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:29.043883 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaaaba5-1def-4311-b7ba-88364c24bb42 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:29.044070 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac57322-73bf-4536-9ccb-41409d604286 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:29.044328 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac57322-73bf-4536-9ccb-41409d604286 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:29.044676 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac57322-73bf-4536-9ccb-41409d604286 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:29.052924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac57322-73bf-4536-9ccb-41409d604286 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:29.055105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac57322-73bf-4536-9ccb-41409d604286 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:29.073477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-0ac57322-73bf-4536-9ccb-41409d604286 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 18:27:29.073477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:29.073477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:29.073477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:29.073477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:29.073477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:29.073477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:29.073477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:29.073477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:29.073477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:29.073477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:29.073477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:29.073477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:29.073477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:29.073477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:29.073477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:29.074510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:29.074510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:29.074510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:29.074510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:29.074510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:29.074510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:29.074510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:29.074510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:29.074510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:29.074510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:29.074510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:29.074510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:29.074510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:29.074510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:29.074510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:29.074510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:29.074510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:29.074510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:29.075434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:29.075434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:29.075434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:29.075434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:29.075434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:29.075434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:29.075434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:29.075434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:29.075434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 18:27:29.075434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:29.075434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 387/754] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Jun 27 18:27:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:29.076025 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e45a9b-9be3-4695-870b-26146f532a5d None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:29.076603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e45a9b-9be3-4695-870b-26146f532a5d None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:29.076603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e45a9b-9be3-4695-870b-26146f532a5d None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:29.077940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 388/755] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:29.082014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e45a9b-9be3-4695-870b-26146f532a5d None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:29.156872 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:29.163577 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaaaba5-1def-4311-b7ba-88364c24bb42 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:29.214085 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698492c-63f5-4093-82ff-e912fba15ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:29.215535 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698492c-63f5-4093-82ff-e912fba15ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:29.215964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698492c-63f5-4093-82ff-e912fba15ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:29.216406 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698492c-63f5-4093-82ff-e912fba15ce8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:29.218080 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e45a9b-9be3-4695-870b-26146f532a5d None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:29.228192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698492c-63f5-4093-82ff-e912fba15ce8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:29.230828 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698492c-63f5-4093-82ff-e912fba15ce8 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:29.234152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:29.252268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-c698492c-63f5-4093-82ff-e912fba15ce8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:29.252268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:29.252268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:29.252268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:29.252268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:29.252268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:29.252268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:29.252268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:29.252268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:29.252268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:29.252268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:29.252268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:29.252268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:29.252268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:29.252268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:29.269933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:29.269933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:29.269933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:29.269933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:29.269933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:29.269933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:29.269933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:29.269933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:29.269933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:29.269933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:29.269933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:29.269933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:29.269933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:29.269933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:29.269933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:29.269933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:29.269933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:29.270926 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:29.270926 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:29.270926 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:29.270926 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:29.270926 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 368/756] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:27:29.270926 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d91c1a-d865-4118-bf79-855c9bf555e7 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:29.270926 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d91c1a-d865-4118-bf79-855c9bf555e7 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:29.270926 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d91c1a-d865-4118-bf79-855c9bf555e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:29.271391 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4de3de-2f25-44b3-9d86-40d78c30de62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:29.271391 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4de3de-2f25-44b3-9d86-40d78c30de62 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:29.271391 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4de3de-2f25-44b3-9d86-40d78c30de62 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:29.271391 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4de3de-2f25-44b3-9d86-40d78c30de62 None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:29.271660 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4de3de-2f25-44b3-9d86-40d78c30de62 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:29.271660 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4de3de-2f25-44b3-9d86-40d78c30de62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:29.271660 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4de3de-2f25-44b3-9d86-40d78c30de62 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:29.279830 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-93d91c1a-d865-4118-bf79-855c9bf555e7 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:29.281249 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-93d91c1a-d865-4118-bf79-855c9bf555e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:29.287657 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 389/757] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Jun 27 18:27:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:29.288589 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4de3de-2f25-44b3-9d86-40d78c30de62 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:29.358315 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 369/758] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:29.359525 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d91c1a-d865-4118-bf79-855c9bf555e7 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:29.424945 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4de3de-2f25-44b3-9d86-40d78c30de62 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:29.433361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:29.472421 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc78421-1093-414c-b0f7-aba5f22708cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:29.473328 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc78421-1093-414c-b0f7-aba5f22708cb None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:29.473686 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc78421-1093-414c-b0f7-aba5f22708cb None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:29.474056 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc78421-1093-414c-b0f7-aba5f22708cb None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:29.474612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d91c1a-d865-4118-bf79-855c9bf555e7 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:29.474953 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc78421-1093-414c-b0f7-aba5f22708cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:29.475328 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc78421-1093-414c-b0f7-aba5f22708cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:29.477044 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc78421-1093-414c-b0f7-aba5f22708cb None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:29.494637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 390/759] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:29.496329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc78421-1093-414c-b0f7-aba5f22708cb None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:29.508644 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:29.520390 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c898f10-30b5-4011-a91a-99cbb481db6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:29.521106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c898f10-30b5-4011-a91a-99cbb481db6a None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:29.521467 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c898f10-30b5-4011-a91a-99cbb481db6a None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:29.521824 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c898f10-30b5-4011-a91a-99cbb481db6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:29.612104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c898f10-30b5-4011-a91a-99cbb481db6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:29.613736 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c898f10-30b5-4011-a91a-99cbb481db6a None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:29.618221 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc78421-1093-414c-b0f7-aba5f22708cb None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:29.662599 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61681d-6c1b-472d-ad99-702fa1f416c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:29.663821 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61681d-6c1b-472d-ad99-702fa1f416c4 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:29.664177 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61681d-6c1b-472d-ad99-702fa1f416c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:29.664549 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef61681d-6c1b-472d-ad99-702fa1f416c4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:29.665288 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61681d-6c1b-472d-ad99-702fa1f416c4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:29.667458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef61681d-6c1b-472d-ad99-702fa1f416c4 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:29.683534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-ef61681d-6c1b-472d-ad99-702fa1f416c4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 18:27:29.683534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:29.683534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:27:29.683534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:27:29.683534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:27:29.683534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:27:29.683534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:27:29.683534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:29.683534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 18:27:29.683534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:29.683534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:29.683534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:29.683534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:29.683534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:29.683534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:29.683534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:29.684700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:29.684700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:29.684700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:29.684700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:29.684700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:29.684700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:29.684700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:29.684700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:27:29.684700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:27:29.684700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:27:29.684700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:27:29.684700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:29.684700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:29.684700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:29.684700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:29.684700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:29.685563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:29.685563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:29.685563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:29.685563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:29.685563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:29.685563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:29.685563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:29.685563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:29.685563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:29.685563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:29.685563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:29.685563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:27:29.685563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:29.685563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 18:27:29.685563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:29.689720 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 391/760] 10.222.0.15 () {56 vars in 1129 bytes} [Mon Jun 27 18:27:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:29.690943 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61681d-6c1b-472d-ad99-702fa1f416c4 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:29.728723 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:29.738560 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 370/761] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:27:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 18:27:29.739721 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c898f10-30b5-4011-a91a-99cbb481db6a None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:29.772637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:29.815993 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef61681d-6c1b-472d-ad99-702fa1f416c4 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:29.853344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f11a6-d239-4dbe-86cb-e39f2f820ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:29.854798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f11a6-d239-4dbe-86cb-e39f2f820ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:29.855371 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f11a6-d239-4dbe-86cb-e39f2f820ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:29.855933 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f11a6-d239-4dbe-86cb-e39f2f820ec4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:29.863729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55f11a6-d239-4dbe-86cb-e39f2f820ec4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:29.865463 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55f11a6-d239-4dbe-86cb-e39f2f820ec4 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:29.865648 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c898f10-30b5-4011-a91a-99cbb481db6a None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:29.881616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-b55f11a6-d239-4dbe-86cb-e39f2f820ec4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:29.881616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:29.881616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:29.881616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:29.881616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:29.881616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:29.881616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:29.881616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:29.881616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:29.881616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:29.881616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:29.881616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:29.881616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:29.881616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:29.881616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:29.882560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:29.882560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:29.882560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:29.882560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:29.882560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:29.882560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:29.882560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:29.882560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:29.882560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:29.882560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:29.882560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:29.882560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:29.882560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:29.882560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:29.882560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:29.882560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:29.882560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:29.883477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:29.883477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:29.883477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:29.883477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:29.883477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:29.883477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:29.883477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:29.883477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:29.886701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 392/762] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:29.887680 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f11a6-d239-4dbe-86cb-e39f2f820ec4 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:29.907720 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a05f5-55b1-4e75-9f6d-4011e116ca0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:29.908759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a05f5-55b1-4e75-9f6d-4011e116ca0b None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:29.909351 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a05f5-55b1-4e75-9f6d-4011e116ca0b None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:29.909872 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a05f5-55b1-4e75-9f6d-4011e116ca0b None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:29.910865 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a05f5-55b1-4e75-9f6d-4011e116ca0b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:29.911366 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a05f5-55b1-4e75-9f6d-4011e116ca0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:29.913100 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a05f5-55b1-4e75-9f6d-4011e116ca0b None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:29.923097 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 371/763] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Jun 27 18:27:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:29.924261 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a05f5-55b1-4e75-9f6d-4011e116ca0b None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:30.002069 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:30.013263 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f11a6-d239-4dbe-86cb-e39f2f820ec4 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:30.035344 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:30.049242 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a05f5-55b1-4e75-9f6d-4011e116ca0b None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:30.057724 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0e524-6e46-4546-998c-7f07250589ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:30.058146 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0e524-6e46-4546-998c-7f07250589ec None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:30.058434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0e524-6e46-4546-998c-7f07250589ec None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:30.058714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0e524-6e46-4546-998c-7f07250589ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:30.072114 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0e524-6e46-4546-998c-7f07250589ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:30.074100 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0e524-6e46-4546-998c-7f07250589ec None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:30.090562 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61930ce-2d3d-45fb-af38-274115316486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:30.091529 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61930ce-2d3d-45fb-af38-274115316486 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:30.092009 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61930ce-2d3d-45fb-af38-274115316486 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:30.092554 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61930ce-2d3d-45fb-af38-274115316486 None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:30.093581 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61930ce-2d3d-45fb-af38-274115316486 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:30.094092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61930ce-2d3d-45fb-af38-274115316486 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:30.095732 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61930ce-2d3d-45fb-af38-274115316486 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:30.095840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-7ee0e524-6e46-4546-998c-7f07250589ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:30.095840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:30.095840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:30.095840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:30.095840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:30.095840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:30.095840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:30.095840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:30.095840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:30.095840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:30.095840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:30.095840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:30.095840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:30.095840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:30.095840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:30.096687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:30.096687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:30.096687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:30.096687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:30.096687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:30.096687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:30.096687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:30.096687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:30.096687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:30.096687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:30.096687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:30.096687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:30.096687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:30.096687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:30.096687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:30.096687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:30.096687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:30.097593 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:30.097593 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:30.097593 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:30.097593 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:30.101799 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 393/764] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:30.102708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0e524-6e46-4546-998c-7f07250589ec None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:30.114694 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 372/765] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:30.115884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61930ce-2d3d-45fb-af38-274115316486 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:30.217906 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0e524-6e46-4546-998c-7f07250589ec None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:30.250548 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61930ce-2d3d-45fb-af38-274115316486 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:30.269501 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078e434-47ae-44b8-941c-e2e00fa79f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:30.270827 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078e434-47ae-44b8-941c-e2e00fa79f13 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:30.271335 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078e434-47ae-44b8-941c-e2e00fa79f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:30.271804 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078e434-47ae-44b8-941c-e2e00fa79f13 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:30.273258 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d078e434-47ae-44b8-941c-e2e00fa79f13 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:30.273774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d078e434-47ae-44b8-941c-e2e00fa79f13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:30.276158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d078e434-47ae-44b8-941c-e2e00fa79f13 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:30.288878 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cce7c1-fb91-4401-9245-51fe5e31d08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:30.288878 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cce7c1-fb91-4401-9245-51fe5e31d08a None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:30.288878 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cce7c1-fb91-4401-9245-51fe5e31d08a None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:30.289363 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cce7c1-fb91-4401-9245-51fe5e31d08a None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:30.289551 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 394/766] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Jun 27 18:27:30 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:30.289898 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cce7c1-fb91-4401-9245-51fe5e31d08a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:30.290212 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cce7c1-fb91-4401-9245-51fe5e31d08a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:30.291017 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078e434-47ae-44b8-941c-e2e00fa79f13 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:30.292121 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cce7c1-fb91-4401-9245-51fe5e31d08a None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:30.302860 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:30.309917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 373/767] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:30.310920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cce7c1-fb91-4401-9245-51fe5e31d08a None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:30.324562 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:30.424365 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cce7c1-fb91-4401-9245-51fe5e31d08a None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:30.431548 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078e434-47ae-44b8-941c-e2e00fa79f13 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:30.460568 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3effda0-1f0b-49b1-8989-fbe56b163798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:30.461236 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3effda0-1f0b-49b1-8989-fbe56b163798 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:30.461520 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3effda0-1f0b-49b1-8989-fbe56b163798 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:30.461796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3effda0-1f0b-49b1-8989-fbe56b163798 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:30.462627 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3effda0-1f0b-49b1-8989-fbe56b163798 None admin] RBAC: Adding query filter params (interface=public, service_id=b0be67912bf0478081972c54f7e9b26b, region_id=RegionOne) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:30.462917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3effda0-1f0b-49b1-8989-fbe56b163798 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:30.465006 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3effda0-1f0b-49b1-8989-fbe56b163798 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:30.477022 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 374/768] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Jun 27 18:27:30 2022] GET /identity/v3/endpoints?service_id=b0be67912bf0478081972c54f7e9b26b&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:30.477022 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3effda0-1f0b-49b1-8989-fbe56b163798 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:30.484952 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5d781-063a-447a-932c-5ab126945d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:30.486709 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5d781-063a-447a-932c-5ab126945d28 None admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:30.487196 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5d781-063a-447a-932c-5ab126945d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:30.488213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5d781-063a-447a-932c-5ab126945d28 None admin] PATH_INFO: `/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/1dddd4efb44c4152bb4151f7715f2223/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:30.491088 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5d781-063a-447a-932c-5ab126945d28 None admin] RBAC: Authorizing `identity:create_grant(project_id=84ecf552f8364884b4aeff3428cb81e4, user_id=1dddd4efb44c4152bb4151f7715f2223, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:30.493457 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5d781-063a-447a-932c-5ab126945d28 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:30.554757 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:30.576029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 395/769] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:27:30 2022] PUT /identity/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/1dddd4efb44c4152bb4151f7715f2223/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:27:30.577184 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5d781-063a-447a-932c-5ab126945d28 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:30.580651 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3effda0-1f0b-49b1-8989-fbe56b163798 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:30.626920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e64b7c-dca5-42cb-9da2-90458dd90153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:30.629587 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e64b7c-dca5-42cb-9da2-90458dd90153 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:30.630153 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e64b7c-dca5-42cb-9da2-90458dd90153 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:30.630703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e64b7c-dca5-42cb-9da2-90458dd90153 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:30.640935 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e64b7c-dca5-42cb-9da2-90458dd90153 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:30.643519 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e64b7c-dca5-42cb-9da2-90458dd90153 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:30.665235 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:30.667990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-33e64b7c-dca5-42cb-9da2-90458dd90153 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:30.667990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:30.667990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:30.667990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:30.667990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:30.667990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:30.667990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:30.667990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:30.667990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:30.667990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:30.667990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:30.667990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:30.667990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:30.667990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:30.667990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:30.668876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:30.668876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:30.668876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:30.668876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:30.668876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:30.668876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:30.668876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:30.668876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:30.668876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:30.668876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:30.668876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:30.668876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:30.668876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:30.668876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:30.668876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:30.668876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:30.668876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:30.669768 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:30.669768 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:30.669768 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:30.669768 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:30.669768 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:30.669768 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:30.669768 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:30.669768 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:30.669768 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 375/770] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:30.676407 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ae96e-9828-47be-b268-8f40ab3c1f1f None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:30.676407 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ae96e-9828-47be-b268-8f40ab3c1f1f None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:30.676407 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ae96e-9828-47be-b268-8f40ab3c1f1f None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:30.676407 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 376/771] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:30.683098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ae96e-9828-47be-b268-8f40ab3c1f1f None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:30.720774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5d781-063a-447a-932c-5ab126945d28 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:30.772868 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa0fa7-0c9c-416d-9b10-7e9d3e26891f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:30.773583 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa0fa7-0c9c-416d-9b10-7e9d3e26891f None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:30.773867 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa0fa7-0c9c-416d-9b10-7e9d3e26891f None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:30.774196 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa0fa7-0c9c-416d-9b10-7e9d3e26891f None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:30.774977 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa0fa7-0c9c-416d-9b10-7e9d3e26891f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:30.776925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa0fa7-0c9c-416d-9b10-7e9d3e26891f None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:30.782797 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 396/772] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Jun 27 18:27:30 2022] GET /identity/v3/services => generated 772 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:30.784085 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff482be-7a29-4e35-9852-22d9145fd0b5 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:30.784376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff482be-7a29-4e35-9852-22d9145fd0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:30.784667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff482be-7a29-4e35-9852-22d9145fd0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:30.802242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-1ff482be-7a29-4e35-9852-22d9145fd0b5 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:30.803512 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-1ff482be-7a29-4e35-9852-22d9145fd0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:30.878038 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 397/773] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:30.881446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ae96e-9828-47be-b268-8f40ab3c1f1f None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:30.915251 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff482be-7a29-4e35-9852-22d9145fd0b5 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:30.927906 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ece293-304f-48a6-a15f-86edcde56370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:30.928964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ece293-304f-48a6-a15f-86edcde56370 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:30.929478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ece293-304f-48a6-a15f-86edcde56370 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:30.929960 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ece293-304f-48a6-a15f-86edcde56370 None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:30.931012 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ece293-304f-48a6-a15f-86edcde56370 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:30.931616 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ece293-304f-48a6-a15f-86edcde56370 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:30.933914 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ece293-304f-48a6-a15f-86edcde56370 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:30.936172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:30.948081 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 377/774] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:30.953630 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ece293-304f-48a6-a15f-86edcde56370 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:30.984348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:31.031280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff482be-7a29-4e35-9852-22d9145fd0b5 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:31.075068 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d298e9-be20-4b37-85a7-723f7f595098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:31.075990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d298e9-be20-4b37-85a7-723f7f595098 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:31.076175 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d298e9-be20-4b37-85a7-723f7f595098 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:31.076446 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d298e9-be20-4b37-85a7-723f7f595098 None admin] PATH_INFO: `/v3/services/network` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:31.077103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d298e9-be20-4b37-85a7-723f7f595098 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:31.078521 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d298e9-be20-4b37-85a7-723f7f595098 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:31.084714 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ece293-304f-48a6-a15f-86edcde56370 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:31.090692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-d7d298e9-be20-4b37-85a7-723f7f595098 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 18:27:31.090692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:31.090692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:27:31.090692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:27:31.090692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:27:31.090692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:27:31.090692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:27:31.090692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:31.090692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 18:27:31.090692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:31.090692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:31.090692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:31.090692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:31.090692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:31.090692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:31.090692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:31.090692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:31.091679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:31.091679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:31.091679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:31.091679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:31.091679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:31.091679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:31.091679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:27:31.091679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:27:31.091679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:27:31.091679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:27:31.091679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:31.091679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:31.091679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:31.091679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:31.091679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:31.091679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:31.092522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:31.092522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:31.092522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:31.092522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:31.092522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:31.092522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:31.092522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:31.092522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:31.092522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:31.092522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:31.092522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:27:31.092522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:31.092522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 18:27:31.092522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:31.096714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 398/775] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:30 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:27:31.098089 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263963b1-58b3-4a36-911c-b737d441ccb6 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:31.098307 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263963b1-58b3-4a36-911c-b737d441ccb6 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:31.098628 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263963b1-58b3-4a36-911c-b737d441ccb6 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:31.100492 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 399/776] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:31.101245 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263963b1-58b3-4a36-911c-b737d441ccb6 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:31.132228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a70ef8-696a-42ad-bcec-9d1a89aa9b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:31.133609 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a70ef8-696a-42ad-bcec-9d1a89aa9b96 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:31.134148 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a70ef8-696a-42ad-bcec-9d1a89aa9b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:31.134611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a70ef8-696a-42ad-bcec-9d1a89aa9b96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:31.136432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a70ef8-696a-42ad-bcec-9d1a89aa9b96 None admin] RBAC: Adding query filter params (role.id=c4bc6e72b9d647c5b2485d52d55f7dd4, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=58e1f1ac6ea14da0be6e9a24d007847a) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:31.137331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a70ef8-696a-42ad-bcec-9d1a89aa9b96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:31.139362 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a70ef8-696a-42ad-bcec-9d1a89aa9b96 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:31.146983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 378/777] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:27:30 2022] GET /identity/v3/role_assignments?user.id=58e1f1ac6ea14da0be6e9a24d007847a&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=c4bc6e72b9d647c5b2485d52d55f7dd4 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:31.149089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d6258-d9b0-4513-b1da-38d6cd38a6ff None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:31.149545 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d6258-d9b0-4513-b1da-38d6cd38a6ff None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:31.150002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d6258-d9b0-4513-b1da-38d6cd38a6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:31.171792 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-276d6258-d9b0-4513-b1da-38d6cd38a6ff None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:31.173521 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-276d6258-d9b0-4513-b1da-38d6cd38a6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:31.224990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263963b1-58b3-4a36-911c-b737d441ccb6 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:31.233518 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:31.248889 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 379/778] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:31.255620 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:31.268789 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74988957-4096-4bea-ac5b-2c45efb08568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:31.269770 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74988957-4096-4bea-ac5b-2c45efb08568 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:31.270233 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74988957-4096-4bea-ac5b-2c45efb08568 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:31.270683 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74988957-4096-4bea-ac5b-2c45efb08568 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:31.271651 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74988957-4096-4bea-ac5b-2c45efb08568 None admin] RBAC: Adding query filter params (name=network) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:31.272126 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74988957-4096-4bea-ac5b-2c45efb08568 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:31.273844 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74988957-4096-4bea-ac5b-2c45efb08568 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:31.285500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d6258-d9b0-4513-b1da-38d6cd38a6ff None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:31.285807 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 400/779] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Jun 27 18:27:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:31.287336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74988957-4096-4bea-ac5b-2c45efb08568 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:31.426784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d6258-d9b0-4513-b1da-38d6cd38a6ff None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:31.469223 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6fa024-98ae-479b-9d4f-e4bb0e144e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:31.470718 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fa024-98ae-479b-9d4f-e4bb0e144e8b None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:31.471154 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fa024-98ae-479b-9d4f-e4bb0e144e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:31.471676 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fa024-98ae-479b-9d4f-e4bb0e144e8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:31.483807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6fa024-98ae-479b-9d4f-e4bb0e144e8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:31.493340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-fb6fa024-98ae-479b-9d4f-e4bb0e144e8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:27:31.493340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:31.493340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:31.493340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:31.493340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:31.493340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:31.493340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:31.493340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:31.493340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:31.493340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:31.493340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:31.493340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:31.493340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:31.493340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:31.493340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:31.493340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:31.495042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:31.495042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:31.495042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:31.495042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:31.495042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:31.495042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:31.495042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:31.495042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:31.495042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:31.495042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:31.495042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:31.495042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:31.495042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:31.495042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:31.495042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:31.495042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:31.496078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:31.496078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:31.496078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:31.496078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:31.496078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:31.496078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:31.496078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:31.496078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:31.496078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:31.496078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:31.496078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:27:31.496078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:31.498633 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 380/780] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:27:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:27:31.504167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6fa024-98ae-479b-9d4f-e4bb0e144e8b None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:31.516803 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74988957-4096-4bea-ac5b-2c45efb08568 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:31.579308 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65691e7d-f0bf-40ba-9ac8-935e2f4e0f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:31.580486 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65691e7d-f0bf-40ba-9ac8-935e2f4e0f19 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:31.581006 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65691e7d-f0bf-40ba-9ac8-935e2f4e0f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:31.581495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65691e7d-f0bf-40ba-9ac8-935e2f4e0f19 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:31.582353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65691e7d-f0bf-40ba-9ac8-935e2f4e0f19 None admin] RBAC: Adding query filter params (type=network) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:31.582824 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65691e7d-f0bf-40ba-9ac8-935e2f4e0f19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:31.584697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65691e7d-f0bf-40ba-9ac8-935e2f4e0f19 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:31.595821 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 401/781] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Jun 27 18:27:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:31.597443 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65691e7d-f0bf-40ba-9ac8-935e2f4e0f19 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:31.659451 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:31.667068 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:31.716646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65691e7d-f0bf-40ba-9ac8-935e2f4e0f19 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:31.761077 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70656e86-5d18-4ec6-b8bf-8e723d1d3ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:31.763186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70656e86-5d18-4ec6-b8bf-8e723d1d3ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:31.763680 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70656e86-5d18-4ec6-b8bf-8e723d1d3ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:31.764076 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70656e86-5d18-4ec6-b8bf-8e723d1d3ec7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:31.765250 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70656e86-5d18-4ec6-b8bf-8e723d1d3ec7 None admin] RBAC: Adding query filter params (interface=public, service_id=2bc17993aaeb465296efc162f2ab3a7d, region_id=RegionOne) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:31.765390 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70656e86-5d18-4ec6-b8bf-8e723d1d3ec7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:31.771029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70656e86-5d18-4ec6-b8bf-8e723d1d3ec7 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:31.777946 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6fa024-98ae-479b-9d4f-e4bb0e144e8b None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:31.793033 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 402/782] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Jun 27 18:27:31 2022] GET /identity/v3/endpoints?service_id=2bc17993aaeb465296efc162f2ab3a7d&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:31.797811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70656e86-5d18-4ec6-b8bf-8e723d1d3ec7 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:31.841553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fc178-5163-4474-81fc-a14b5b196cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:31.842520 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fc178-5163-4474-81fc-a14b5b196cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:31.842951 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fc178-5163-4474-81fc-a14b5b196cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:31.843688 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fc178-5163-4474-81fc-a14b5b196cc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:31.844586 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fc178-5163-4474-81fc-a14b5b196cc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:31.845041 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fc178-5163-4474-81fc-a14b5b196cc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:31.847024 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fc178-5163-4474-81fc-a14b5b196cc2 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:31.858291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 381/783] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:27:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:31.863811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fc178-5163-4474-81fc-a14b5b196cc2 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:31.986237 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fc178-5163-4474-81fc-a14b5b196cc2 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:32.000502 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:32.009070 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70656e86-5d18-4ec6-b8bf-8e723d1d3ec7 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:32.026851 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49dedba-d83e-4df2-82cf-9f370d7775bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:32.027618 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49dedba-d83e-4df2-82cf-9f370d7775bf None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:32.027900 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49dedba-d83e-4df2-82cf-9f370d7775bf None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:32.028172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49dedba-d83e-4df2-82cf-9f370d7775bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:32.043634 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:32.100301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611bcb6-56c9-4424-98ca-5b56d7949add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:32.100993 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611bcb6-56c9-4424-98ca-5b56d7949add None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:32.101281 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611bcb6-56c9-4424-98ca-5b56d7949add None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:32.101557 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611bcb6-56c9-4424-98ca-5b56d7949add None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:32.102277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611bcb6-56c9-4424-98ca-5b56d7949add None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:32.103757 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611bcb6-56c9-4424-98ca-5b56d7949add None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:32.117776 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 403/784] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Jun 27 18:27:31 2022] GET /identity/v3/services => generated 772 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:32.185312 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49dedba-d83e-4df2-82cf-9f370d7775bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:32.186917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49dedba-d83e-4df2-82cf-9f370d7775bf None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:32.218350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-a49dedba-d83e-4df2-82cf-9f370d7775bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:32.218350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:32.218350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:32.218350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:32.218350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:32.218350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:32.218350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:32.218350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:32.218350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:32.218350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:32.218350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:32.218350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:32.218350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:32.218350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:32.218350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:32.219342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:32.219342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:32.219342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:32.219342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:32.219342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:32.219342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:32.219342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:32.219342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:32.219342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:32.219342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:32.219342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:32.219342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:32.219342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:32.219342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:32.219342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:32.219342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:32.219342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:32.220223 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:32.220223 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:32.220223 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:32.220223 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:32.220223 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 382/785] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:27:32.227506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611bcb6-56c9-4424-98ca-5b56d7949add None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:32.359595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611bcb6-56c9-4424-98ca-5b56d7949add None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:32.396684 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9358d6-bac5-4ee1-9906-a9d831eae994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:32.397344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9358d6-bac5-4ee1-9906-a9d831eae994 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:32.397630 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9358d6-bac5-4ee1-9906-a9d831eae994 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:32.397905 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9358d6-bac5-4ee1-9906-a9d831eae994 None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:32.398693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9358d6-bac5-4ee1-9906-a9d831eae994 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:32.399000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9358d6-bac5-4ee1-9906-a9d831eae994 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:32.400380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9358d6-bac5-4ee1-9906-a9d831eae994 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:32.412629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 404/786] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:32.418679 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49dedba-d83e-4df2-82cf-9f370d7775bf None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:32.556545 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49dedba-d83e-4df2-82cf-9f370d7775bf None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:32.611219 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca02c0-20a7-48f8-9164-ac07618f6f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:32.611219 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca02c0-20a7-48f8-9164-ac07618f6f64 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:32.611219 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca02c0-20a7-48f8-9164-ac07618f6f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:32.611219 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca02c0-20a7-48f8-9164-ac07618f6f64 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:32.618951 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca02c0-20a7-48f8-9164-ac07618f6f64 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:32.620925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca02c0-20a7-48f8-9164-ac07618f6f64 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:32.643647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-d6ca02c0-20a7-48f8-9164-ac07618f6f64 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 18:27:32.643647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:32.643647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:32.643647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:32.643647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:32.643647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:32.643647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:32.643647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:32.643647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:32.643647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:32.643647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:32.643647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:32.643647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:32.643647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:32.643647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:32.643647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:32.644681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:32.644681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:32.644681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:32.644681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:32.644681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:32.644681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:32.644681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:32.644681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:32.644681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:32.644681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:32.644681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:32.644681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:32.644681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:32.644681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:32.644681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:32.644681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:32.644681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:32.644681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:32.645630 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:32.645630 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:32.645630 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:32.645630 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:32.645630 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:32.645630 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:32.645630 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:32.645630 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:32.645630 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 18:27:32.645630 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:32.648347 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 383/787] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:27:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:32.655489 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9358d6-bac5-4ee1-9906-a9d831eae994 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:32.732214 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:32.817250 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9358d6-bac5-4ee1-9906-a9d831eae994 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:32.861024 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38110e-1696-4f96-bef6-0e45685e08c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:32.861750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38110e-1696-4f96-bef6-0e45685e08c5 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:32.861979 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38110e-1696-4f96-bef6-0e45685e08c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:32.862259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38110e-1696-4f96-bef6-0e45685e08c5 None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:32.863139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e38110e-1696-4f96-bef6-0e45685e08c5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:32.863451 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e38110e-1696-4f96-bef6-0e45685e08c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:32.864895 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e38110e-1696-4f96-bef6-0e45685e08c5 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:32.886831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 405/788] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Jun 27 18:27:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:32.892523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca02c0-20a7-48f8-9164-ac07618f6f64 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:32.945153 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:33.039046 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca02c0-20a7-48f8-9164-ac07618f6f64 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:33.074549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc2129d-3449-4007-99b3-3b4f705bbce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:33.075346 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2129d-3449-4007-99b3-3b4f705bbce6 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:33.075624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2129d-3449-4007-99b3-3b4f705bbce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:33.075899 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2129d-3449-4007-99b3-3b4f705bbce6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:33.086922 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc2129d-3449-4007-99b3-3b4f705bbce6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:33.088544 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc2129d-3449-4007-99b3-3b4f705bbce6 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:33.107293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-6fc2129d-3449-4007-99b3-3b4f705bbce6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:33.107293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:33.107293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:33.107293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:33.107293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:33.107293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:33.107293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:33.107293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:33.107293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:33.107293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:33.107293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:33.107293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:33.107293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:33.107293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:33.107293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:33.108273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:33.108273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:33.108273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:33.108273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:33.108273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:33.108273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:33.108273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:33.108273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:33.108273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:33.108273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:33.108273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:33.108273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:33.108273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:33.108273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:33.108273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:33.108273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:33.108273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:33.109162 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:33.109162 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:33.109162 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:33.109162 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:33.113239 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 384/789] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:33.114678 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38110e-1696-4f96-bef6-0e45685e08c5 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:33.137984 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43653717-d623-487f-9a8e-2b141b6b17cf None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:33.138376 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43653717-d623-487f-9a8e-2b141b6b17cf None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:33.138727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43653717-d623-487f-9a8e-2b141b6b17cf None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:33.140230 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 385/790] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:27:33.146102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e956c81-1a6a-4eee-9bcf-9588fbfb626b None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:33.146356 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e956c81-1a6a-4eee-9bcf-9588fbfb626b None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:33.146686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e956c81-1a6a-4eee-9bcf-9588fbfb626b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:33.165763 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-8e956c81-1a6a-4eee-9bcf-9588fbfb626b None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:33.167650 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-8e956c81-1a6a-4eee-9bcf-9588fbfb626b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:33.270680 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 386/791] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:33.271088 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:33.274021 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38110e-1696-4f96-bef6-0e45685e08c5 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:33.279761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2359cc0-33f2-4372-ae39-eb8c3168884f None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:33.280050 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2359cc0-33f2-4372-ae39-eb8c3168884f None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:33.280388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2359cc0-33f2-4372-ae39-eb8c3168884f None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:33.281915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 387/792] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:33 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:33.292494 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b1516-cd4e-4b0d-ad4c-ead00029c97f None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:33.292788 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b1516-cd4e-4b0d-ad4c-ead00029c97f None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:33.293052 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b1516-cd4e-4b0d-ad4c-ead00029c97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:33.305735 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:33.310664 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c59182-ad9e-424f-890f-e357157d63e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:33.311439 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c59182-ad9e-424f-890f-e357157d63e0 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:33.311731 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c59182-ad9e-424f-890f-e357157d63e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:33.312011 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c59182-ad9e-424f-890f-e357157d63e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:33.312860 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c59182-ad9e-424f-890f-e357157d63e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:33.313282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c59182-ad9e-424f-890f-e357157d63e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:33.314754 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c59182-ad9e-424f-890f-e357157d63e0 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:33.321698 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-e49b1516-cd4e-4b0d-ad4c-ead00029c97f None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:33.323490 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-e49b1516-cd4e-4b0d-ad4c-ead00029c97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:33.328442 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 406/793] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:33.329330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c59182-ad9e-424f-890f-e357157d63e0 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:33.411856 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 388/794] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:33.412637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b1516-cd4e-4b0d-ad4c-ead00029c97f None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:33.474447 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c59182-ad9e-424f-890f-e357157d63e0 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:33.511155 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef30dc-fd37-435f-8b1d-464bb78c078b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:33.512273 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef30dc-fd37-435f-8b1d-464bb78c078b None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:33.512772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef30dc-fd37-435f-8b1d-464bb78c078b None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:33.513283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef30dc-fd37-435f-8b1d-464bb78c078b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:33.526535 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef30dc-fd37-435f-8b1d-464bb78c078b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:33.534771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-72ef30dc-fd37-435f-8b1d-464bb78c078b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:27:33.534771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:33.534771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:33.534771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:33.534771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:33.534771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:33.534771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:33.534771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:33.534771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:33.534771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:33.534771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:33.534771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:33.534771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:33.534771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:33.534771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:33.534771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:33.535732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:33.535732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:33.535732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:33.535732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:33.535732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:33.535732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:33.535732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:33.535732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:33.535732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:33.535732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:33.535732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:33.535732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:33.535732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:33.535732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:33.535732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:33.535732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:33.536561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:33.536561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:33.536561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:33.536561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:33.536561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:33.536561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:33.536561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:33.536561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:33.536561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:33.536561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:33.536561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:27:33.536561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:33.536561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 407/795] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:27:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:33.537264 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef30dc-fd37-435f-8b1d-464bb78c078b None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:33.543868 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b1516-cd4e-4b0d-ad4c-ead00029c97f None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:33.560874 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:33.581282 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13a85d-2106-408e-bcbd-3dcf01385bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:33.582075 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13a85d-2106-408e-bcbd-3dcf01385bfd None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:33.582355 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13a85d-2106-408e-bcbd-3dcf01385bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:33.582688 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13a85d-2106-408e-bcbd-3dcf01385bfd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:33.584660 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:33.591330 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13a85d-2106-408e-bcbd-3dcf01385bfd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:33.592801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13a85d-2106-408e-bcbd-3dcf01385bfd None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:33.608965 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-da13a85d-2106-408e-bcbd-3dcf01385bfd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:33.608965 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:33.608965 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:33.608965 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:33.608965 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:33.608965 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:33.608965 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:33.608965 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:33.608965 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:33.608965 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:33.608965 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:33.608965 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:33.608965 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:33.608965 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:33.608965 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:33.614340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:33.614340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:33.614340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:33.614340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:33.614340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:33.614340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:33.614340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:33.614340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:33.614340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:33.614340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:33.614340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:33.614340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:33.614340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:33.614340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:33.614340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:33.614340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:33.614340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:33.617130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:33.617130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:33.617130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:33.617130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:33.617130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:33.617130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:33.617130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:33.617130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:33.617130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 389/796] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:33.617130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13a85d-2106-408e-bcbd-3dcf01385bfd None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:33.645990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef30dc-fd37-435f-8b1d-464bb78c078b None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:33.682526 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9134617-8253-412d-8494-670406fda6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:33.683493 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9134617-8253-412d-8494-670406fda6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:33.683785 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9134617-8253-412d-8494-670406fda6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:33.684067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9134617-8253-412d-8494-670406fda6a7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:33.684761 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9134617-8253-412d-8494-670406fda6a7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:33.686278 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9134617-8253-412d-8494-670406fda6a7 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:33.699128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-f9134617-8253-412d-8494-670406fda6a7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 18:27:33.699128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:33.699128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:27:33.699128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:27:33.699128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:27:33.699128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:27:33.699128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:27:33.699128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:33.699128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 18:27:33.699128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:33.699128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:33.699128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:33.699128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:33.699128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:33.699128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:33.699128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:33.701325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:33.701325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:33.701325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:33.701325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:33.701325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:33.701325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:33.701325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:33.701325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:27:33.701325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:27:33.701325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:27:33.701325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:27:33.701325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:33.701325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:33.701325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:33.701325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:33.701325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:33.702650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:33.702650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:33.702650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:33.702650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:33.702650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:33.702650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:33.702650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:33.702650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:33.702650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:33.702650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:33.702650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:33.702650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:27:33.702650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:33.702650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 18:27:33.702650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:33.702650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 408/797] 10.222.0.15 () {56 vars in 1129 bytes} [Mon Jun 27 18:27:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:33.703693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9134617-8253-412d-8494-670406fda6a7 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:33.741858 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13a85d-2106-408e-bcbd-3dcf01385bfd None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:33.812688 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe132565-3c12-4011-857e-815978ba98c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:33.813553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe132565-3c12-4011-857e-815978ba98c2 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:33.813930 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe132565-3c12-4011-857e-815978ba98c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:33.814303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe132565-3c12-4011-857e-815978ba98c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:33.815196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe132565-3c12-4011-857e-815978ba98c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:33.815600 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe132565-3c12-4011-857e-815978ba98c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:33.817666 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe132565-3c12-4011-857e-815978ba98c2 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:33.825898 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:33.836454 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 390/798] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:27:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:33.836454 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe132565-3c12-4011-857e-815978ba98c2 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:33.855594 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9134617-8253-412d-8494-670406fda6a7 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:33.916234 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:33.922795 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec34463-729e-416a-b7b8-1fb4b8f7d1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:33.923462 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec34463-729e-416a-b7b8-1fb4b8f7d1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:33.930078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec34463-729e-416a-b7b8-1fb4b8f7d1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:33.930078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec34463-729e-416a-b7b8-1fb4b8f7d1c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:33.930078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec34463-729e-416a-b7b8-1fb4b8f7d1c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:33.930078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec34463-729e-416a-b7b8-1fb4b8f7d1c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:33.930078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec34463-729e-416a-b7b8-1fb4b8f7d1c0 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:33.950458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 409/799] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:33.956720 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657cff8-5a26-4b5a-abad-6cdaaee0f4d2 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:33.957013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657cff8-5a26-4b5a-abad-6cdaaee0f4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:33.957292 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657cff8-5a26-4b5a-abad-6cdaaee0f4d2 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:33.959114 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 410/800] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:33.959844 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657cff8-5a26-4b5a-abad-6cdaaee0f4d2 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:33.981950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe132565-3c12-4011-857e-815978ba98c2 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:34.023040 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a04d6-d03e-4e1c-9f0e-890771f325cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:34.023695 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a04d6-d03e-4e1c-9f0e-890771f325cb None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:34.023974 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a04d6-d03e-4e1c-9f0e-890771f325cb None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:34.024243 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a04d6-d03e-4e1c-9f0e-890771f325cb None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:34.024985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a04d6-d03e-4e1c-9f0e-890771f325cb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:34.025293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a04d6-d03e-4e1c-9f0e-890771f325cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:34.027316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a04d6-d03e-4e1c-9f0e-890771f325cb None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:34.038759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 391/801] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Jun 27 18:27:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:34.042610 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a04d6-d03e-4e1c-9f0e-890771f325cb None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:34.123092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657cff8-5a26-4b5a-abad-6cdaaee0f4d2 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:34.169879 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be58ff4-32b7-42ca-aa8e-b30af96049e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:34.171264 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be58ff4-32b7-42ca-aa8e-b30af96049e9 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:34.171544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be58ff4-32b7-42ca-aa8e-b30af96049e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:34.172105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be58ff4-32b7-42ca-aa8e-b30af96049e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:34.186464 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be58ff4-32b7-42ca-aa8e-b30af96049e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:34.188289 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be58ff4-32b7-42ca-aa8e-b30af96049e9 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:34.191113 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:34.192982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a04d6-d03e-4e1c-9f0e-890771f325cb None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:34.212341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-7be58ff4-32b7-42ca-aa8e-b30af96049e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:34.212341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:34.212341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:34.212341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:34.212341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:34.212341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:34.212341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:34.212341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:34.212341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:34.212341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:34.212341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:34.212341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:34.212341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:34.212341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:34.212341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:34.213422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:34.213422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:34.213422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:34.213422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:34.213422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:34.213422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:34.213422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:34.213422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:34.213422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:34.213422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:34.213422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:34.213422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:34.213422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:34.213422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:34.213422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:34.213422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:34.213422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:34.214760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:34.214760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:34.214760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:34.214760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:34.214760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 411/802] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:34.215671 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f150e-a21e-4e9d-9f73-a61519de724b None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:34.215934 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f150e-a21e-4e9d-9f73-a61519de724b None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:34.216211 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f150e-a21e-4e9d-9f73-a61519de724b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:34.225942 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:34.242871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66490c11-e38a-43fb-b7a7-aefe6f544864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:34.243242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-f36f150e-a21e-4e9d-9f73-a61519de724b None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:34.243770 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66490c11-e38a-43fb-b7a7-aefe6f544864 None admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:34.244050 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66490c11-e38a-43fb-b7a7-aefe6f544864 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:34.244865 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66490c11-e38a-43fb-b7a7-aefe6f544864 None admin] PATH_INFO: `/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/a1529c9a45434a19968e924797688f06/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:34.244968 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-f36f150e-a21e-4e9d-9f73-a61519de724b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:34.246966 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66490c11-e38a-43fb-b7a7-aefe6f544864 None admin] RBAC: Authorizing `identity:create_grant(project_id=84ecf552f8364884b4aeff3428cb81e4, user_id=a1529c9a45434a19968e924797688f06, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:34.248407 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66490c11-e38a-43fb-b7a7-aefe6f544864 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:34.315063 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 392/803] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:27:34 2022] PUT /identity/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/a1529c9a45434a19968e924797688f06/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:27:34.316257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66490c11-e38a-43fb-b7a7-aefe6f544864 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:34.338504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 412/804] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:34.338504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f150e-a21e-4e9d-9f73-a61519de724b None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:34.483921 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f150e-a21e-4e9d-9f73-a61519de724b None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:34.485500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66490c11-e38a-43fb-b7a7-aefe6f544864 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:34.520183 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:34.534990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b87b5f-aa63-4cd2-ab82-35d297cb56c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:34.535808 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b87b5f-aa63-4cd2-ab82-35d297cb56c2 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:34.536093 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b87b5f-aa63-4cd2-ab82-35d297cb56c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:34.536368 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b87b5f-aa63-4cd2-ab82-35d297cb56c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:34.537102 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b87b5f-aa63-4cd2-ab82-35d297cb56c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:34.537413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b87b5f-aa63-4cd2-ab82-35d297cb56c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:34.539180 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b87b5f-aa63-4cd2-ab82-35d297cb56c2 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:34.540342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100523e8-04e6-4359-b04b-4c200f484bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:34.541008 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100523e8-04e6-4359-b04b-4c200f484bee None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:34.541774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100523e8-04e6-4359-b04b-4c200f484bee None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:34.542047 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100523e8-04e6-4359-b04b-4c200f484bee None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:34.542793 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100523e8-04e6-4359-b04b-4c200f484bee None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:34.543107 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100523e8-04e6-4359-b04b-4c200f484bee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:34.544553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100523e8-04e6-4359-b04b-4c200f484bee None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:34.552650 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 393/805] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Jun 27 18:27:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:34.555268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100523e8-04e6-4359-b04b-4c200f484bee None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:34.558727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 413/806] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:34.559800 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b87b5f-aa63-4cd2-ab82-35d297cb56c2 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:34.617010 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:34.677882 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100523e8-04e6-4359-b04b-4c200f484bee None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:34.683029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b87b5f-aa63-4cd2-ab82-35d297cb56c2 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:34.716342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca64ef77-3f32-478a-bc0a-b85b54e01d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:34.717133 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64ef77-3f32-478a-bc0a-b85b54e01d46 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:34.717417 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64ef77-3f32-478a-bc0a-b85b54e01d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:34.717687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64ef77-3f32-478a-bc0a-b85b54e01d46 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:34.718274 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c45c90-2284-4b54-ada8-9295f95aed90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:34.718968 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c45c90-2284-4b54-ada8-9295f95aed90 None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:34.719253 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c45c90-2284-4b54-ada8-9295f95aed90 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:34.719632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c45c90-2284-4b54-ada8-9295f95aed90 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:34.720257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c45c90-2284-4b54-ada8-9295f95aed90 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:34.721626 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c45c90-2284-4b54-ada8-9295f95aed90 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:34.729488 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca64ef77-3f32-478a-bc0a-b85b54e01d46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:34.737703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-ca64ef77-3f32-478a-bc0a-b85b54e01d46 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:34.737703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:34.737703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:34.737703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:34.737703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:34.737703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:34.737703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:34.737703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:34.737703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:34.737703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:34.737703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:34.737703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:34.737703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:34.737703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:34.737703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:34.737703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:34.739155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:34.739155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:34.739155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:34.739155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:34.739155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:34.739155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:34.739155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:34.739155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:34.739155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:34.739155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:34.739155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:34.739155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:34.739155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:34.739155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:34.739155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:34.739155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:34.740106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:34.740106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:34.740106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:34.740106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:34.740106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:34.740106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:34.740106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:34.740106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:34.740106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:34.740106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:34.740106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:34.740106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:34.744078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 394/807] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:27:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:34.745628 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca64ef77-3f32-478a-bc0a-b85b54e01d46 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:34.839803 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 414/808] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Jun 27 18:27:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:27:34.841298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cbeba-8e50-42ba-bb9d-fe5107af05ac None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:34.841681 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cbeba-8e50-42ba-bb9d-fe5107af05ac None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:34.841970 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cbeba-8e50-42ba-bb9d-fe5107af05ac None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:34.843699 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 415/809] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:34.844454 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cbeba-8e50-42ba-bb9d-fe5107af05ac None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:34.853365 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca64ef77-3f32-478a-bc0a-b85b54e01d46 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:34.863746 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:34.903574 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425066b-17bf-4163-a3ed-4014218cf82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:34.904553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425066b-17bf-4163-a3ed-4014218cf82a None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:34.904888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425066b-17bf-4163-a3ed-4014218cf82a None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:34.905170 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425066b-17bf-4163-a3ed-4014218cf82a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:34.912206 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:34.915636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425066b-17bf-4163-a3ed-4014218cf82a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:34.917298 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1425066b-17bf-4163-a3ed-4014218cf82a None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:34.933823 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-1425066b-17bf-4163-a3ed-4014218cf82a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 18:27:34.933823 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:34.933823 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:34.933823 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:34.933823 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:34.933823 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:34.933823 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:34.933823 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:34.933823 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:34.933823 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:34.933823 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:34.933823 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:34.933823 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:34.933823 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:34.933823 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:34.933823 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:34.939550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:34.939550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:34.939550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:34.939550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:34.939550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:34.939550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:34.939550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:34.939550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:34.939550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:34.939550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:34.939550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:34.939550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:34.939550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:34.939550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:34.939550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:34.939550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:34.939550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:34.939550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:34.940518 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:34.940518 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:34.940518 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:34.940518 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:34.940518 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:34.940518 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:34.940518 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:34.940518 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:34.940518 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 18:27:34.940518 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:34.941070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 395/810] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Jun 27 18:27:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:34.942217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e937aa-9e4e-4fbb-80d1-7280c5cd8d97 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:34.942332 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e937aa-9e4e-4fbb-80d1-7280c5cd8d97 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:34.946404 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e937aa-9e4e-4fbb-80d1-7280c5cd8d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:34.963061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-75e937aa-9e4e-4fbb-80d1-7280c5cd8d97 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:34.964492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-75e937aa-9e4e-4fbb-80d1-7280c5cd8d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:35.002761 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cbeba-8e50-42ba-bb9d-fe5107af05ac None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:35.051295 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 396/811] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:35.052140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e937aa-9e4e-4fbb-80d1-7280c5cd8d97 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:35.053007 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37d66f-c57d-493d-a8a8-ab5feddb3c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:35.053902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37d66f-c57d-493d-a8a8-ab5feddb3c34 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:35.054609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37d66f-c57d-493d-a8a8-ab5feddb3c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:35.055935 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37d66f-c57d-493d-a8a8-ab5feddb3c34 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:35.057457 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37d66f-c57d-493d-a8a8-ab5feddb3c34 None admin] RBAC: Adding query filter params (name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:35.058090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37d66f-c57d-493d-a8a8-ab5feddb3c34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:35.059719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37d66f-c57d-493d-a8a8-ab5feddb3c34 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:35.072485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 416/812] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Jun 27 18:27:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:35.079425 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37d66f-c57d-493d-a8a8-ab5feddb3c34 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:35.164422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:35.176307 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e937aa-9e4e-4fbb-80d1-7280c5cd8d97 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:35.201768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37d66f-c57d-493d-a8a8-ab5feddb3c34 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:35.204328 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:35.224325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210aef4a-be37-453a-a55f-6cabb15b573a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:35.224998 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210aef4a-be37-453a-a55f-6cabb15b573a None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:35.225284 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210aef4a-be37-453a-a55f-6cabb15b573a None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:35.225565 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210aef4a-be37-453a-a55f-6cabb15b573a None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:35.226431 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210aef4a-be37-453a-a55f-6cabb15b573a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:35.226803 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210aef4a-be37-453a-a55f-6cabb15b573a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:35.229079 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210aef4a-be37-453a-a55f-6cabb15b573a None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:35.247164 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14152c-68ec-439b-8f94-9ec907cc81b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:35.248016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14152c-68ec-439b-8f94-9ec907cc81b7 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:35.248789 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14152c-68ec-439b-8f94-9ec907cc81b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:35.249088 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14152c-68ec-439b-8f94-9ec907cc81b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:35.252472 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 397/813] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Jun 27 18:27:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:35.253531 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210aef4a-be37-453a-a55f-6cabb15b573a None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:35.263880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14152c-68ec-439b-8f94-9ec907cc81b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:35.265696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14152c-68ec-439b-8f94-9ec907cc81b7 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:35.286055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-ec14152c-68ec-439b-8f94-9ec907cc81b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:35.286055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:35.286055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:35.286055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:35.286055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:35.286055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:35.286055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:35.286055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:35.286055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:35.286055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:35.286055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:35.286055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:35.286055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:35.286055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:35.286055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:35.287282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:35.287282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:35.287282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:35.287282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:35.287282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:35.287282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:35.287282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:35.287282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:35.287282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:35.287282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:35.287282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:35.287282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:35.287282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:35.287282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:35.287282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:35.287282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:35.287282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:35.288223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:35.288223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:35.288223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:35.288223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:35.295211 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 417/814] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:35.296456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14152c-68ec-439b-8f94-9ec907cc81b7 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:35.384495 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210aef4a-be37-453a-a55f-6cabb15b573a None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:35.408945 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14152c-68ec-439b-8f94-9ec907cc81b7 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:35.434634 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642aa5a9-a597-4b7d-bcea-75e0a46cd777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:35.435701 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642aa5a9-a597-4b7d-bcea-75e0a46cd777 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:35.435990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642aa5a9-a597-4b7d-bcea-75e0a46cd777 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:35.436266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642aa5a9-a597-4b7d-bcea-75e0a46cd777 None admin] PATH_INFO: `/v3/services/network` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:35.436925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642aa5a9-a597-4b7d-bcea-75e0a46cd777 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:35.438991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642aa5a9-a597-4b7d-bcea-75e0a46cd777 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:35.450771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb22061-53ac-4e12-b147-94b9c5ed43f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:35.451731 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb22061-53ac-4e12-b147-94b9c5ed43f2 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:35.452233 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb22061-53ac-4e12-b147-94b9c5ed43f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:35.452716 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb22061-53ac-4e12-b147-94b9c5ed43f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:35.454512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-642aa5a9-a597-4b7d-bcea-75e0a46cd777 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 18:27:35.454512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:35.454512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:27:35.454512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:27:35.454512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:27:35.454512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:27:35.454512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:27:35.454512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:35.454512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 18:27:35.454512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:35.454512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:35.454512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:35.454512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:35.454512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:35.454512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:35.454512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:35.454512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:35.455506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:35.455506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:35.455506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:35.455506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:35.455506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:35.455506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:35.455506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:27:35.455506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:27:35.455506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:27:35.455506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:27:35.455506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:35.455506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:35.455506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:35.455506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:35.455506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:35.455506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:35.456342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:35.456342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:35.456342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:35.456342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:35.456342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:35.456342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:35.456342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:35.456342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:35.456342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:35.456342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:35.456342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:27:35.456342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:35.456342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 18:27:35.456342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:35.460736 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 398/815] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:35 2022] GET /identity/v3/services/network => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:35.462609 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642aa5a9-a597-4b7d-bcea-75e0a46cd777 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:35.482213 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:35.489274 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:35.550110 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb22061-53ac-4e12-b147-94b9c5ed43f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:35.552359 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb22061-53ac-4e12-b147-94b9c5ed43f2 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:35.578681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642aa5a9-a597-4b7d-bcea-75e0a46cd777 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:35.626006 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d604de-cadf-41ff-8681-665999261bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:35.627648 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d604de-cadf-41ff-8681-665999261bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:35.628170 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d604de-cadf-41ff-8681-665999261bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:35.628860 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d604de-cadf-41ff-8681-665999261bb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:35.630271 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d604de-cadf-41ff-8681-665999261bb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:35.631065 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d604de-cadf-41ff-8681-665999261bb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:35.633222 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d604de-cadf-41ff-8681-665999261bb8 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:35.652923 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 399/816] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:35.653798 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d604de-cadf-41ff-8681-665999261bb8 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:35.684598 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 418/817] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:27:35 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 18:27:35.685625 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb22061-53ac-4e12-b147-94b9c5ed43f2 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:35.747416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:35.748402 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:35.788432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d604de-cadf-41ff-8681-665999261bb8 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:35.797634 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb22061-53ac-4e12-b147-94b9c5ed43f2 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:35.830655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7200c182-22df-4d65-9583-6acb33049fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:35.832181 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200c182-22df-4d65-9583-6acb33049fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:35.832655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200c182-22df-4d65-9583-6acb33049fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:35.833177 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200c182-22df-4d65-9583-6acb33049fb5 None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:35.834232 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7200c182-22df-4d65-9583-6acb33049fb5 None admin] RBAC: Adding query filter params (name=network) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:35.834724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7200c182-22df-4d65-9583-6acb33049fb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:35.835934 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ddf31-5d24-4ad0-99a5-cbfa17153792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:35.836591 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7200c182-22df-4d65-9583-6acb33049fb5 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:35.836887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ddf31-5d24-4ad0-99a5-cbfa17153792 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:35.837240 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ddf31-5d24-4ad0-99a5-cbfa17153792 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:35.837589 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ddf31-5d24-4ad0-99a5-cbfa17153792 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:35.848216 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 400/818] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Jun 27 18:27:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:35.849023 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ddf31-5d24-4ad0-99a5-cbfa17153792 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:35.849501 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7200c182-22df-4d65-9583-6acb33049fb5 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:35.850661 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ddf31-5d24-4ad0-99a5-cbfa17153792 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:35.867966 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-e99ddf31-5d24-4ad0-99a5-cbfa17153792 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 18:27:35.867966 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:35.867966 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:35.867966 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:35.867966 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:35.867966 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:35.867966 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:35.867966 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:35.867966 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:35.867966 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:35.867966 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:35.867966 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:35.867966 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:35.867966 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:35.867966 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:35.867966 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:35.868944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:35.868944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:35.868944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:35.868944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:35.868944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:35.868944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:35.868944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:35.868944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:35.868944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:35.868944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:35.868944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:35.868944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:35.868944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:35.868944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:35.868944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:35.868944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:35.868944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:35.868944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:35.870513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:35.870513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:35.870513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:35.870513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:35.870513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:35.870513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:35.870513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:35.870513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:35.870513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 18:27:35.870513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:35.870513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 419/819] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Jun 27 18:27:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:35.875222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ddf31-5d24-4ad0-99a5-cbfa17153792 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:35.974363 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7200c182-22df-4d65-9583-6acb33049fb5 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:35.990650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ddf31-5d24-4ad0-99a5-cbfa17153792 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:36.010098 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:36.013797 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06545bbf-ea95-47d3-9f61-2e5e9820fe82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:36.014743 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06545bbf-ea95-47d3-9f61-2e5e9820fe82 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:36.015125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06545bbf-ea95-47d3-9f61-2e5e9820fe82 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:36.015500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06545bbf-ea95-47d3-9f61-2e5e9820fe82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:36.018670 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:36.028481 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06545bbf-ea95-47d3-9f61-2e5e9820fe82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:36.030051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06545bbf-ea95-47d3-9f61-2e5e9820fe82 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:36.036883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3230329-017b-446c-bca6-e11a1947d78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:36.038227 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3230329-017b-446c-bca6-e11a1947d78f None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:36.038802 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3230329-017b-446c-bca6-e11a1947d78f None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:36.039222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3230329-017b-446c-bca6-e11a1947d78f None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:36.040587 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3230329-017b-446c-bca6-e11a1947d78f None admin] RBAC: Adding query filter params (type=network) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:36.041233 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3230329-017b-446c-bca6-e11a1947d78f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:36.042893 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3230329-017b-446c-bca6-e11a1947d78f None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:36.053113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-06545bbf-ea95-47d3-9f61-2e5e9820fe82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:36.053113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:36.053113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:36.053113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:36.053113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:36.053113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:36.053113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:36.053113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:36.053113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:36.053113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:36.053113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:36.053113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:36.053113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:36.053113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:36.053113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:36.055944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:36.055944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:36.055944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:36.055944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:36.055944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:36.055944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:36.055944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:36.055944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:36.055944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:36.055944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:36.055944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:36.055944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:36.055944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:36.055944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:36.055944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:36.055944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:36.055944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:36.057024 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:36.057024 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:36.057024 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:36.057024 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:36.057257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 420/821] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Jun 27 18:27:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:36.057257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3230329-017b-446c-bca6-e11a1947d78f None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:36.057636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 401/821] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:36.058667 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06545bbf-ea95-47d3-9f61-2e5e9820fe82 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:36.172967 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3230329-017b-446c-bca6-e11a1947d78f None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:36.202776 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06545bbf-ea95-47d3-9f61-2e5e9820fe82 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:36.209513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987550c-857d-4662-8240-7cbc993c5b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:36.210263 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987550c-857d-4662-8240-7cbc993c5b8b None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:36.210810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987550c-857d-4662-8240-7cbc993c5b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:36.211164 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987550c-857d-4662-8240-7cbc993c5b8b None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:36.212228 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987550c-857d-4662-8240-7cbc993c5b8b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:36.212611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987550c-857d-4662-8240-7cbc993c5b8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:36.214277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987550c-857d-4662-8240-7cbc993c5b8b None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:36.233742 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 421/822] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Jun 27 18:27:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:36.238861 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987550c-857d-4662-8240-7cbc993c5b8b None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:36.248366 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c9800-3bf3-4b6c-8a04-4118b958447e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:36.249255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c9800-3bf3-4b6c-8a04-4118b958447e None admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:36.249706 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c9800-3bf3-4b6c-8a04-4118b958447e None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:36.250541 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c9800-3bf3-4b6c-8a04-4118b958447e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:36.251478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c9800-3bf3-4b6c-8a04-4118b958447e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:36.253559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c9800-3bf3-4b6c-8a04-4118b958447e None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:36.277472 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:36.341829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987550c-857d-4662-8240-7cbc993c5b8b None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:36.358927 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 402/823] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Jun 27 18:27:36 2022] POST /identity/v3/endpoints => generated 327 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:27:36.360253 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c9800-3bf3-4b6c-8a04-4118b958447e None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:36.425358 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb05b0-6de7-470b-9ab9-904af0d95e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:36.425949 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb05b0-6de7-470b-9ab9-904af0d95e0d None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:36.426232 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb05b0-6de7-470b-9ab9-904af0d95e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:36.426532 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb05b0-6de7-470b-9ab9-904af0d95e0d None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:36.427278 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb05b0-6de7-470b-9ab9-904af0d95e0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:36.433635 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb05b0-6de7-470b-9ab9-904af0d95e0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:36.435240 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb05b0-6de7-470b-9ab9-904af0d95e0d None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:36.453421 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 422/824] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:36.459281 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87234cbf-d0c8-4929-857a-3f68dcc54929 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:36.459564 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87234cbf-d0c8-4929-857a-3f68dcc54929 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:36.459849 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87234cbf-d0c8-4929-857a-3f68dcc54929 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:36.460100 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:36.466469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 423/825] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:36.466469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87234cbf-d0c8-4929-857a-3f68dcc54929 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:36.537556 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c9800-3bf3-4b6c-8a04-4118b958447e None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:36.581079 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73367d4b-2bee-49f5-a20f-8ed068984993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:36.581881 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73367d4b-2bee-49f5-a20f-8ed068984993 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:36.582159 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73367d4b-2bee-49f5-a20f-8ed068984993 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:36.582729 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73367d4b-2bee-49f5-a20f-8ed068984993 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:36.595486 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73367d4b-2bee-49f5-a20f-8ed068984993 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:36.596985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73367d4b-2bee-49f5-a20f-8ed068984993 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:36.597417 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:36.613228 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87234cbf-d0c8-4929-857a-3f68dcc54929 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:36.620351 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-73367d4b-2bee-49f5-a20f-8ed068984993 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:36.620351 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:36.620351 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:36.620351 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:36.620351 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:36.620351 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:36.620351 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:36.620351 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:36.620351 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:36.620351 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:36.620351 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:36.620351 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:36.620351 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:36.620351 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:36.620351 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:36.621310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:36.621310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:36.621310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:36.621310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:36.621310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:36.621310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:36.621310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:36.621310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:36.621310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:36.621310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:36.621310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:36.621310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:36.621310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:36.621310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:36.621310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:36.621310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:36.621310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:36.622195 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:36.622195 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:36.622195 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:36.622195 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:36.622195 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 403/826] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:36.623462 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e1070-d89d-4bec-8855-9ce66570812e None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:36.623570 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e1070-d89d-4bec-8855-9ce66570812e None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:36.623840 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e1070-d89d-4bec-8855-9ce66570812e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:36.643694 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-1e0e1070-d89d-4bec-8855-9ce66570812e None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:36.644889 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-1e0e1070-d89d-4bec-8855-9ce66570812e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:36.660800 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50fb6b8-dca8-47b5-8ba3-725b86ed1236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:36.661545 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fb6b8-dca8-47b5-8ba3-725b86ed1236 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:36.661827 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fb6b8-dca8-47b5-8ba3-725b86ed1236 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:36.662103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fb6b8-dca8-47b5-8ba3-725b86ed1236 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:36.669546 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50fb6b8-dca8-47b5-8ba3-725b86ed1236 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:36.671077 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50fb6b8-dca8-47b5-8ba3-725b86ed1236 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:36.685513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-c50fb6b8-dca8-47b5-8ba3-725b86ed1236 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:36.685513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:36.685513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:36.685513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:36.685513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:36.685513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:36.685513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:36.685513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:36.685513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:36.685513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:36.685513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:36.685513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:36.685513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:36.685513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:36.685513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:36.686434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:36.686434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:36.686434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:36.686434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:36.686434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:36.686434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:36.686434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:36.686434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:36.686434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:36.686434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:36.686434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:36.686434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:36.686434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:36.686434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:36.686434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:36.686434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:36.686434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:36.687333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:36.687333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:36.687333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:36.687333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:36.687333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:36.687333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:36.687333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:36.687333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:36.690109 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 424/827] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:36.691627 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50fb6b8-dca8-47b5-8ba3-725b86ed1236 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:36.719045 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 404/828] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:36.720043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e1070-d89d-4bec-8855-9ce66570812e None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:36.755433 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:36.808881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50fb6b8-dca8-47b5-8ba3-725b86ed1236 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:36.832445 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e1070-d89d-4bec-8855-9ce66570812e None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:36.844455 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab4221-e51f-4a18-a9e7-0440f08b778b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:36.845116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab4221-e51f-4a18-a9e7-0440f08b778b None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:36.845400 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab4221-e51f-4a18-a9e7-0440f08b778b None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:36.845677 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab4221-e51f-4a18-a9e7-0440f08b778b None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:36.846426 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab4221-e51f-4a18-a9e7-0440f08b778b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:36.846739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab4221-e51f-4a18-a9e7-0440f08b778b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:36.848111 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab4221-e51f-4a18-a9e7-0440f08b778b None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:36.865452 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 425/829] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:36.865452 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab4221-e51f-4a18-a9e7-0440f08b778b None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:36.874175 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8f261-c127-4172-92a2-59aaeba72d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:36.874869 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8f261-c127-4172-92a2-59aaeba72d2f None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:36.875149 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8f261-c127-4172-92a2-59aaeba72d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:36.875429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8f261-c127-4172-92a2-59aaeba72d2f None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:36.876203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb8f261-c127-4172-92a2-59aaeba72d2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:36.876510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb8f261-c127-4172-92a2-59aaeba72d2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:36.877935 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb8f261-c127-4172-92a2-59aaeba72d2f None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:36.881685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:36.895045 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 405/830] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:36.895045 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8f261-c127-4172-92a2-59aaeba72d2f None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:36.976927 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab4221-e51f-4a18-a9e7-0440f08b778b None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:36.983842 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:36.998641 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8f261-c127-4172-92a2-59aaeba72d2f None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:37.015735 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8510bb05-0f91-415b-a7d6-343549dd0923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:37.016968 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510bb05-0f91-415b-a7d6-343549dd0923 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:37.017413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510bb05-0f91-415b-a7d6-343549dd0923 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:37.017914 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510bb05-0f91-415b-a7d6-343549dd0923 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:37.032502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8510bb05-0f91-415b-a7d6-343549dd0923 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:37.041736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-8510bb05-0f91-415b-a7d6-343549dd0923 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:27:37.041736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:37.041736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:37.041736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:37.041736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:37.041736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:37.041736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:37.041736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:37.041736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:37.041736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:37.041736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:37.041736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:37.041736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:37.041736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:37.041736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:37.041736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:37.042798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:37.042798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:37.042798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:37.042798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:37.042798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:37.042798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:37.042798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:37.042798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:37.042798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:37.042798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:37.042798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:37.042798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:37.042798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:37.042798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:37.042798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:37.042798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:37.043746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:37.043746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:37.043746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:37.043746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:37.043746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:37.043746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:37.043746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:37.043746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:37.043746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:37.043746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:37.043746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:27:37.043746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:37.044397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d30f9a-828a-45e2-b922-d0ba5f5a0ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:37.044489 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 426/831] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:27:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:37.044567 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d30f9a-828a-45e2-b922-d0ba5f5a0ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:37.044567 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d30f9a-828a-45e2-b922-d0ba5f5a0ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:37.044691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8510bb05-0f91-415b-a7d6-343549dd0923 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:37.044760 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d30f9a-828a-45e2-b922-d0ba5f5a0ec1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:37.054086 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d30f9a-828a-45e2-b922-d0ba5f5a0ec1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:37.055558 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d30f9a-828a-45e2-b922-d0ba5f5a0ec1 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:37.069336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-c3d30f9a-828a-45e2-b922-d0ba5f5a0ec1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:37.069336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:37.069336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:37.069336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:37.069336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:37.069336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:37.069336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:37.069336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:37.069336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:37.069336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:37.069336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:37.069336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:37.069336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:37.069336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:37.069336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:37.070255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:37.070255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:37.070255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:37.070255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:37.070255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:37.070255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:37.070255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:37.070255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:37.070255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:37.070255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:37.070255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:37.070255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:37.070255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:37.070255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:37.070255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:37.070255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:37.070255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:37.071209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:37.071209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:37.071209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:37.071209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:37.071209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:37.071209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:37.071209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:37.071209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:37.071209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 406/832] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:37.071877 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8634a67-1f6e-49a2-b793-d03aa9ee523a None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:37.071949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8634a67-1f6e-49a2-b793-d03aa9ee523a None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:37.072839 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8634a67-1f6e-49a2-b793-d03aa9ee523a None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:37.074186 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 407/833] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:37.074742 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8634a67-1f6e-49a2-b793-d03aa9ee523a None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:37.120680 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:37.150112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8510bb05-0f91-415b-a7d6-343549dd0923 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:37.176564 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8634a67-1f6e-49a2-b793-d03aa9ee523a None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:37.184142 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fab48e-e715-42c8-a9cf-05825aade682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:37.184789 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fab48e-e715-42c8-a9cf-05825aade682 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:37.185075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fab48e-e715-42c8-a9cf-05825aade682 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:37.185354 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fab48e-e715-42c8-a9cf-05825aade682 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:37.186184 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fab48e-e715-42c8-a9cf-05825aade682 None admin] RBAC: Adding query filter params (role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=1dddd4efb44c4152bb4151f7715f2223) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:37.186524 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fab48e-e715-42c8-a9cf-05825aade682 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:37.187954 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fab48e-e715-42c8-a9cf-05825aade682 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:37.193674 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 427/834] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:27:37 2022] GET /identity/v3/role_assignments?user.id=1dddd4efb44c4152bb4151f7715f2223&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:37.194465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fab48e-e715-42c8-a9cf-05825aade682 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:37.216363 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c7f5c-b560-4584-8021-84724fafbe4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:37.217512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c7f5c-b560-4584-8021-84724fafbe4b None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:37.217976 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c7f5c-b560-4584-8021-84724fafbe4b None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:37.218457 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c7f5c-b560-4584-8021-84724fafbe4b None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:37.219413 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c7f5c-b560-4584-8021-84724fafbe4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:37.220020 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c7f5c-b560-4584-8021-84724fafbe4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:37.221634 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c7f5c-b560-4584-8021-84724fafbe4b None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:37.233094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 408/835] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:27:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:37.234788 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064ca74-9358-45db-a555-5825a1ab73e1 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:37.235068 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064ca74-9358-45db-a555-5825a1ab73e1 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:37.235337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064ca74-9358-45db-a555-5825a1ab73e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:37.255053 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-e064ca74-9358-45db-a555-5825a1ab73e1 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:37.256377 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-e064ca74-9358-45db-a555-5825a1ab73e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:37.258862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:37.316036 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fab48e-e715-42c8-a9cf-05825aade682 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:37.342658 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 409/836] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:37.342658 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064ca74-9358-45db-a555-5825a1ab73e1 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:37.368368 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af800a-2830-41ba-90e5-b0c23b09834a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:37.369032 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af800a-2830-41ba-90e5-b0c23b09834a None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:37.369977 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af800a-2830-41ba-90e5-b0c23b09834a None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:37.370264 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af800a-2830-41ba-90e5-b0c23b09834a None admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:37.371915 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af800a-2830-41ba-90e5-b0c23b09834a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:37.372256 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af800a-2830-41ba-90e5-b0c23b09834a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:37.373680 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af800a-2830-41ba-90e5-b0c23b09834a None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:37.377494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:37.389188 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 428/837] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:37.390497 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af800a-2830-41ba-90e5-b0c23b09834a None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:37.459454 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064ca74-9358-45db-a555-5825a1ab73e1 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:37.494414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86856126-8992-4017-96a5-b5e7fa02c702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:37.495072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86856126-8992-4017-96a5-b5e7fa02c702 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:37.495354 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86856126-8992-4017-96a5-b5e7fa02c702 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:37.495626 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86856126-8992-4017-96a5-b5e7fa02c702 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:37.499886 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af800a-2830-41ba-90e5-b0c23b09834a None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:37.502215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:37.506353 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86856126-8992-4017-96a5-b5e7fa02c702 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:37.507816 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86856126-8992-4017-96a5-b5e7fa02c702 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:37.525917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-86856126-8992-4017-96a5-b5e7fa02c702 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:37.525917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:37.525917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:37.525917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:37.525917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:37.525917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:37.525917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:37.525917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:37.525917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:37.525917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:37.525917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:37.525917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:37.525917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:37.525917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:37.525917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:37.527066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:37.527066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:37.527066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:37.527066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:37.527066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:37.527066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:37.527066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:37.527066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:37.527066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:37.527066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:37.527066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:37.527066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:37.527066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:37.527066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:37.527066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:37.527066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:37.527066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:37.527952 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:37.527952 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:37.527952 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:37.527952 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:37.527952 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 410/838] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:37.527952 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86856126-8992-4017-96a5-b5e7fa02c702 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:37.542321 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5ac9a-ee90-4573-a040-82ff976f7930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:37.543145 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5ac9a-ee90-4573-a040-82ff976f7930 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:37.543432 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5ac9a-ee90-4573-a040-82ff976f7930 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:37.543715 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5ac9a-ee90-4573-a040-82ff976f7930 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:37.544373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5ac9a-ee90-4573-a040-82ff976f7930 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:37.545739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5ac9a-ee90-4573-a040-82ff976f7930 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:37.558153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-3fc5ac9a-ee90-4573-a040-82ff976f7930 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 18:27:37.558153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:37.558153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:27:37.558153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:27:37.558153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:27:37.558153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:27:37.558153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:27:37.558153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:37.558153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 18:27:37.558153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:37.558153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:37.558153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:37.558153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:37.558153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:37.558153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:37.558153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:37.558153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:37.559223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:37.559223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:37.559223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:37.559223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:37.559223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:37.559223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:37.559223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:27:37.559223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:27:37.559223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:27:37.559223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:27:37.559223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:37.559223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:37.559223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:37.559223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:37.559223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:37.559223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:37.560067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:37.560067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:37.560067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:37.560067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:37.560067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:37.560067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:37.560067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:37.560067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:37.560067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:37.560067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:37.560067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:27:37.560067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:37.560067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 18:27:37.560067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:37.560067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 429/839] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Jun 27 18:27:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:37.560897 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5ac9a-ee90-4573-a040-82ff976f7930 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:37.629287 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:37.640391 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86856126-8992-4017-96a5-b5e7fa02c702 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:37.674604 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5ac9a-ee90-4573-a040-82ff976f7930 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:37.676551 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c02d73-e321-4308-a35b-073c0a11f497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:37.677437 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c02d73-e321-4308-a35b-073c0a11f497 None admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:37.677718 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c02d73-e321-4308-a35b-073c0a11f497 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:37.677991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c02d73-e321-4308-a35b-073c0a11f497 None admin] PATH_INFO: `/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/58e1f1ac6ea14da0be6e9a24d007847a/roles/c4bc6e72b9d647c5b2485d52d55f7dd4` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:37.693659 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c02d73-e321-4308-a35b-073c0a11f497 None admin] RBAC: Authorizing `identity:create_grant(project_id=84ecf552f8364884b4aeff3428cb81e4, user_id=58e1f1ac6ea14da0be6e9a24d007847a, role_id=c4bc6e72b9d647c5b2485d52d55f7dd4)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:37.695125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c02d73-e321-4308-a35b-073c0a11f497 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:37.711932 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca33a31-63cc-4898-a9ad-e93d4b1098d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:37.713928 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca33a31-63cc-4898-a9ad-e93d4b1098d4 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:37.713928 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca33a31-63cc-4898-a9ad-e93d4b1098d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:37.713928 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca33a31-63cc-4898-a9ad-e93d4b1098d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:37.714247 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca33a31-63cc-4898-a9ad-e93d4b1098d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:37.714576 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca33a31-63cc-4898-a9ad-e93d4b1098d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:37.715951 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca33a31-63cc-4898-a9ad-e93d4b1098d4 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:37.728346 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 430/840] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:37.729657 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca33a31-63cc-4898-a9ad-e93d4b1098d4 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:37.751989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 411/841] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:27:37 2022] PUT /identity/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/58e1f1ac6ea14da0be6e9a24d007847a/roles/c4bc6e72b9d647c5b2485d52d55f7dd4 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:27:37.754367 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c02d73-e321-4308-a35b-073c0a11f497 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:37.829266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:37.865784 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca33a31-63cc-4898-a9ad-e93d4b1098d4 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:37.891380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:37.907135 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f2f93-080d-4c92-80f3-6dc5842e279d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:37.907805 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f2f93-080d-4c92-80f3-6dc5842e279d None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:37.908147 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f2f93-080d-4c92-80f3-6dc5842e279d None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:37.908422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f2f93-080d-4c92-80f3-6dc5842e279d None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:37.909430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f2f93-080d-4c92-80f3-6dc5842e279d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:37.909430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f2f93-080d-4c92-80f3-6dc5842e279d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:37.910948 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f2f93-080d-4c92-80f3-6dc5842e279d None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:37.918456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 431/842] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Jun 27 18:27:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:37.924179 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f2f93-080d-4c92-80f3-6dc5842e279d None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:37.925297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c02d73-e321-4308-a35b-073c0a11f497 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:37.982478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed685f2-0c6f-450a-82fa-6e1134d030e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:37.983290 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed685f2-0c6f-450a-82fa-6e1134d030e4 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:37.983569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed685f2-0c6f-450a-82fa-6e1134d030e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:37.983841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed685f2-0c6f-450a-82fa-6e1134d030e4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:37.993788 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed685f2-0c6f-450a-82fa-6e1134d030e4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:37.997237 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed685f2-0c6f-450a-82fa-6e1134d030e4 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:38.018279 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-fed685f2-0c6f-450a-82fa-6e1134d030e4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 18:27:38.018279 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:38.018279 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:38.018279 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:38.018279 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:38.018279 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:38.018279 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:38.018279 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:38.018279 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:38.018279 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:38.018279 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:38.018279 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:38.018279 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:38.018279 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:38.018279 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:38.018279 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:38.019484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:38.019484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:38.019484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:38.019484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:38.019484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:38.019484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:38.019484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:38.019484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:38.019484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:38.019484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:38.019484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:38.019484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:38.019484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:38.019484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:38.019484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:38.019484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:38.019484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:38.019484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:38.020661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:38.020661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:38.020661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:38.020661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:38.020661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:38.020661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:38.020661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:38.020661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:38.020661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 18:27:38.020661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:38.020661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 412/843] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:27:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:38.027505 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed685f2-0c6f-450a-82fa-6e1134d030e4 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:38.058408 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f2f93-080d-4c92-80f3-6dc5842e279d None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:38.083830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3df2a-2711-4d8c-90e9-8a94d8c91689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:38.084575 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3df2a-2711-4d8c-90e9-8a94d8c91689 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:38.084924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3df2a-2711-4d8c-90e9-8a94d8c91689 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:38.085260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3df2a-2711-4d8c-90e9-8a94d8c91689 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:38.086039 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3df2a-2711-4d8c-90e9-8a94d8c91689 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:38.086424 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3df2a-2711-4d8c-90e9-8a94d8c91689 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:38.088428 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3df2a-2711-4d8c-90e9-8a94d8c91689 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:38.094037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 432/844] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Jun 27 18:27:37 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:38.139997 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:38.143741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed685f2-0c6f-450a-82fa-6e1134d030e4 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:38.184005 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849fcfd7-4d99-4b02-b929-aaf15a0b7048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:38.184854 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fcfd7-4d99-4b02-b929-aaf15a0b7048 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:38.185231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fcfd7-4d99-4b02-b929-aaf15a0b7048 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:38.185599 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fcfd7-4d99-4b02-b929-aaf15a0b7048 None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:38.186587 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849fcfd7-4d99-4b02-b929-aaf15a0b7048 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:38.186995 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849fcfd7-4d99-4b02-b929-aaf15a0b7048 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:38.188619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849fcfd7-4d99-4b02-b929-aaf15a0b7048 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:38.210890 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 413/845] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Jun 27 18:27:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:38.219621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3df2a-2711-4d8c-90e9-8a94d8c91689 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:38.280525 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:38.332687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3df2a-2711-4d8c-90e9-8a94d8c91689 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:38.369577 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ec0f9-5a16-43b2-87b9-349d07ec07d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:38.370246 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ec0f9-5a16-43b2-87b9-349d07ec07d7 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:38.370565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ec0f9-5a16-43b2-87b9-349d07ec07d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:38.370847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ec0f9-5a16-43b2-87b9-349d07ec07d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:38.444158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ec0f9-5a16-43b2-87b9-349d07ec07d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:38.445631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ec0f9-5a16-43b2-87b9-349d07ec07d7 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:38.613669 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:38.669033 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 433/846] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:27:38 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 18:27:38.675457 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849fcfd7-4d99-4b02-b929-aaf15a0b7048 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:38.779494 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849fcfd7-4d99-4b02-b929-aaf15a0b7048 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:38.814278 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785340dc-3f75-4462-a6cc-8ec248b2beb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:38.815096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785340dc-3f75-4462-a6cc-8ec248b2beb2 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:38.815428 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785340dc-3f75-4462-a6cc-8ec248b2beb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:38.815705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785340dc-3f75-4462-a6cc-8ec248b2beb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:38.830154 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785340dc-3f75-4462-a6cc-8ec248b2beb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:38.831928 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785340dc-3f75-4462-a6cc-8ec248b2beb2 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:38.850197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-785340dc-3f75-4462-a6cc-8ec248b2beb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:38.850197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:38.850197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:38.850197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:38.850197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:38.850197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:38.850197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:38.850197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:38.850197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:38.850197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:38.850197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:38.850197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:38.850197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:38.850197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:38.850197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:38.851286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:38.851286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:38.851286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:38.851286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:38.851286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:38.851286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:38.851286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:38.851286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:38.851286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:38.851286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:38.851286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:38.851286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:38.851286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:38.851286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:38.851286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:38.851286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:38.851286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:38.852172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:38.852172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:38.852172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:38.852172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:38.854893 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 414/847] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:38.860503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ec0f9-5a16-43b2-87b9-349d07ec07d7 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:38.966440 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ec0f9-5a16-43b2-87b9-349d07ec07d7 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:39.004781 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd854929-ace0-4168-b8ca-0aadc67dbf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:39.005442 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd854929-ace0-4168-b8ca-0aadc67dbf2b None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:39.005723 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd854929-ace0-4168-b8ca-0aadc67dbf2b None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:39.006002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd854929-ace0-4168-b8ca-0aadc67dbf2b None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:39.006769 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd854929-ace0-4168-b8ca-0aadc67dbf2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:39.007075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd854929-ace0-4168-b8ca-0aadc67dbf2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:39.008445 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd854929-ace0-4168-b8ca-0aadc67dbf2b None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:39.020820 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 434/848] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:39.026801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785340dc-3f75-4462-a6cc-8ec248b2beb2 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:39.039665 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:39.136415 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785340dc-3f75-4462-a6cc-8ec248b2beb2 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:39.170968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512f053-612a-4764-8ac8-c6f0d5e3e388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:39.172080 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512f053-612a-4764-8ac8-c6f0d5e3e388 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:39.172586 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512f053-612a-4764-8ac8-c6f0d5e3e388 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:39.173049 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512f053-612a-4764-8ac8-c6f0d5e3e388 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:39.179998 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7512f053-612a-4764-8ac8-c6f0d5e3e388 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:39.181727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7512f053-612a-4764-8ac8-c6f0d5e3e388 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:39.195500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-7512f053-612a-4764-8ac8-c6f0d5e3e388 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:39.195500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:39.195500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:39.195500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:39.195500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:39.195500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:39.195500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:39.195500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:39.195500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:39.195500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:39.195500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:39.195500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:39.195500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:39.195500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:39.195500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:39.196371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:39.196371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:39.196371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:39.196371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:39.196371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:39.196371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:39.196371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:39.196371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:39.196371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:39.196371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:39.196371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:39.196371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:39.196371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:39.196371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:39.196371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:39.196371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:39.196371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:39.197849 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:39.197849 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:39.197849 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:39.197849 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:39.197849 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:39.197849 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:39.197849 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:39.197849 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:39.200354 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 415/849] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:39.210402 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd854929-ace0-4168-b8ca-0aadc67dbf2b None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:39.217760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:39.311385 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24906f-42dd-44fa-918c-c4167ce09dc3 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:39.312019 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24906f-42dd-44fa-918c-c4167ce09dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:39.312495 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24906f-42dd-44fa-918c-c4167ce09dc3 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:39.314324 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 416/850] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:39.314705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd854929-ace0-4168-b8ca-0aadc67dbf2b None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:39.319252 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e98350-1849-473a-b1e1-c878a855e392 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:39.319546 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e98350-1849-473a-b1e1-c878a855e392 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:39.319824 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e98350-1849-473a-b1e1-c878a855e392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:39.340015 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-25e98350-1849-473a-b1e1-c878a855e392 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:39.341248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-25e98350-1849-473a-b1e1-c878a855e392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:39.353910 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81afbc3-985d-4208-8f5a-68308efe66f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:39.355005 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81afbc3-985d-4208-8f5a-68308efe66f3 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:39.355005 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81afbc3-985d-4208-8f5a-68308efe66f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:39.355264 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81afbc3-985d-4208-8f5a-68308efe66f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:39.356067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81afbc3-985d-4208-8f5a-68308efe66f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:39.356639 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81afbc3-985d-4208-8f5a-68308efe66f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:39.358239 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81afbc3-985d-4208-8f5a-68308efe66f3 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:39.372500 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 435/851] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:39.378357 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81afbc3-985d-4208-8f5a-68308efe66f3 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:39.420775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:39.434738 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 417/852] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:39.450202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e98350-1849-473a-b1e1-c878a855e392 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:39.458562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:39.495716 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81afbc3-985d-4208-8f5a-68308efe66f3 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:39.532273 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2154f-faef-4058-865c-38014b5faa7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:39.532924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2154f-faef-4058-865c-38014b5faa7b None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:39.533210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2154f-faef-4058-865c-38014b5faa7b None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:39.533492 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2154f-faef-4058-865c-38014b5faa7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:39.534330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b2154f-faef-4058-865c-38014b5faa7b None admin] RBAC: Adding query filter params (role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=a1529c9a45434a19968e924797688f06) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:39.534666 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b2154f-faef-4058-865c-38014b5faa7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:39.536109 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b2154f-faef-4058-865c-38014b5faa7b None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:39.550409 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 436/853] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:27:39 2022] GET /identity/v3/role_assignments?user.id=a1529c9a45434a19968e924797688f06&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:39.576262 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e98350-1849-473a-b1e1-c878a855e392 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:39.614816 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b11356-34c4-4de9-9908-ca87d2aaa21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:39.615727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b11356-34c4-4de9-9908-ca87d2aaa21f None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:39.616040 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b11356-34c4-4de9-9908-ca87d2aaa21f None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:39.616319 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b11356-34c4-4de9-9908-ca87d2aaa21f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:39.616983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b11356-34c4-4de9-9908-ca87d2aaa21f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:39.618377 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b11356-34c4-4de9-9908-ca87d2aaa21f None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:39.632031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-21b11356-34c4-4de9-9908-ca87d2aaa21f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 18:27:39.632031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:39.632031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:27:39.632031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:27:39.632031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:27:39.632031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:27:39.632031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:27:39.632031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:39.632031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 18:27:39.632031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:39.632031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:39.632031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:39.632031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:39.632031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:39.632031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:39.632031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:39.633221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:39.633221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:39.633221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:39.633221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:39.633221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:39.633221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:39.633221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:39.633221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:27:39.633221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:27:39.633221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:27:39.633221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:27:39.633221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:39.633221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:39.633221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:39.633221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:39.633221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:39.634059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:39.634059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:39.634059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:39.634059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:39.634059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:39.634059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:39.634059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:39.634059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:39.634059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:39.634059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:39.634059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:39.634059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:27:39.634059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:39.634059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 18:27:39.634059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:39.636790 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 418/854] 10.222.0.15 () {56 vars in 1132 bytes} [Mon Jun 27 18:27:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:39.642759 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2154f-faef-4058-865c-38014b5faa7b None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:39.763808 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2154f-faef-4058-865c-38014b5faa7b None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:39.795730 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:39.800879 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ffb77-e976-4fe8-989c-81a568cca570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:39.801547 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ffb77-e976-4fe8-989c-81a568cca570 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:39.801834 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ffb77-e976-4fe8-989c-81a568cca570 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:39.802113 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ffb77-e976-4fe8-989c-81a568cca570 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:39.802861 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ffb77-e976-4fe8-989c-81a568cca570 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:39.803169 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ffb77-e976-4fe8-989c-81a568cca570 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:39.804557 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ffb77-e976-4fe8-989c-81a568cca570 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:39.809703 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 437/855] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Jun 27 18:27:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:39.815506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b11356-34c4-4de9-9908-ca87d2aaa21f None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:39.883792 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:39.930409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b11356-34c4-4de9-9908-ca87d2aaa21f None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:39.962804 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762deb9e-1c61-4fab-9580-14f0396d1dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:39.963562 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762deb9e-1c61-4fab-9580-14f0396d1dae None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:39.963845 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762deb9e-1c61-4fab-9580-14f0396d1dae None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:39.964118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762deb9e-1c61-4fab-9580-14f0396d1dae None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:39.964825 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762deb9e-1c61-4fab-9580-14f0396d1dae None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:39.965123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762deb9e-1c61-4fab-9580-14f0396d1dae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:39.966514 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762deb9e-1c61-4fab-9580-14f0396d1dae None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:39.971774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 419/856] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Jun 27 18:27:39 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:39.985706 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077882c2-285d-4e16-aed8-00f936e139e6 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:39.986020 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077882c2-285d-4e16-aed8-00f936e139e6 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:39.986272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077882c2-285d-4e16-aed8-00f936e139e6 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:39.988035 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 438/857] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:39.993756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98f7c4-3be4-4156-a2fb-f3c1403406da None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:39.994191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98f7c4-3be4-4156-a2fb-f3c1403406da None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:39.994410 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98f7c4-3be4-4156-a2fb-f3c1403406da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:40.013418 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-7b98f7c4-3be4-4156-a2fb-f3c1403406da None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:40.014593 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-7b98f7c4-3be4-4156-a2fb-f3c1403406da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:40.096117 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 420/858] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:40.122202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077882c2-285d-4e16-aed8-00f936e139e6 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:40.248973 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077882c2-285d-4e16-aed8-00f936e139e6 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:40.286310 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a0a4b-d09d-4721-9326-8d3c177d66eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:40.287043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a0a4b-d09d-4721-9326-8d3c177d66eb None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:40.287378 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a0a4b-d09d-4721-9326-8d3c177d66eb None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:40.287652 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a0a4b-d09d-4721-9326-8d3c177d66eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:40.300119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a0a4b-d09d-4721-9326-8d3c177d66eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:40.308603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-879a0a4b-d09d-4721-9326-8d3c177d66eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:40.308603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:40.308603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:40.308603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:40.308603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:40.308603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:40.308603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:40.308603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:40.308603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:40.308603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:40.308603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:40.308603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:40.308603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:40.308603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:40.308603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:40.308603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:40.309562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:40.309562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:40.309562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:40.309562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:40.309562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:40.309562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:40.309562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:40.309562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:40.309562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:40.309562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:40.309562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:40.309562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:40.309562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:40.309562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:40.309562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:40.309562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:40.310935 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:40.310935 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:40.310935 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:40.310935 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:40.310935 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:40.310935 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:40.310935 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:40.310935 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:40.310935 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:40.310935 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:40.310935 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:27:40.310935 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:40.310935 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 439/859] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:27:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:40.317943 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b98f7c4-3be4-4156-a2fb-f3c1403406da None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:40.333074 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3b2e1-0f32-4c78-b00b-c77192e67103 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:40.333374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3b2e1-0f32-4c78-b00b-c77192e67103 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:40.333646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3b2e1-0f32-4c78-b00b-c77192e67103 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:40.336552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 440/860] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:27:40.342813 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcd1e1-ca1d-487e-af12-35c337f61153 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:40.343099 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcd1e1-ca1d-487e-af12-35c337f61153 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:40.343377 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcd1e1-ca1d-487e-af12-35c337f61153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:40.365524 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-c2dcd1e1-ca1d-487e-af12-35c337f61153 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:40.367039 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c2dcd1e1-ca1d-487e-af12-35c337f61153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:40.383287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:40.427458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:40.437303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b98f7c4-3be4-4156-a2fb-f3c1403406da None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:40.454754 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 441/861] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:40.470876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcd1e1-ca1d-487e-af12-35c337f61153 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:40.479524 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8e82c-40cc-45a0-86a1-a514d9130b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:40.481042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8e82c-40cc-45a0-86a1-a514d9130b43 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:40.481732 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8e82c-40cc-45a0-86a1-a514d9130b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:40.482160 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8e82c-40cc-45a0-86a1-a514d9130b43 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:40.483554 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8e82c-40cc-45a0-86a1-a514d9130b43 None admin] RBAC: Adding query filter params (name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:40.484018 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8e82c-40cc-45a0-86a1-a514d9130b43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:40.486247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8e82c-40cc-45a0-86a1-a514d9130b43 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:40.498524 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 421/862] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Jun 27 18:27:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:40.504115 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8e82c-40cc-45a0-86a1-a514d9130b43 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:40.603157 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcd1e1-ca1d-487e-af12-35c337f61153 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:40.629447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8e82c-40cc-45a0-86a1-a514d9130b43 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:40.650300 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778e698-2b41-4df9-8a28-e6f0af5c7d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:40.651262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778e698-2b41-4df9-8a28-e6f0af5c7d22 None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:40.651720 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778e698-2b41-4df9-8a28-e6f0af5c7d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:40.652166 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778e698-2b41-4df9-8a28-e6f0af5c7d22 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:40.653065 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a778e698-2b41-4df9-8a28-e6f0af5c7d22 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:40.654652 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a778e698-2b41-4df9-8a28-e6f0af5c7d22 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:40.655275 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:40.671396 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a84a11-274f-45b8-b70c-4d79942a4d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:40.672272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a84a11-274f-45b8-b70c-4d79942a4d55 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:40.672625 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a84a11-274f-45b8-b70c-4d79942a4d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:40.672970 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a84a11-274f-45b8-b70c-4d79942a4d55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:40.683896 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a84a11-274f-45b8-b70c-4d79942a4d55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:40.685369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a84a11-274f-45b8-b70c-4d79942a4d55 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:40.704320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-f4a84a11-274f-45b8-b70c-4d79942a4d55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:40.704320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:40.704320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:40.704320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:40.704320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:40.704320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:40.704320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:40.704320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:40.704320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:40.704320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:40.704320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:40.704320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:40.704320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:40.704320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:40.704320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:40.705266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:40.705266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:40.705266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:40.705266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:40.705266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:40.705266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:40.705266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:40.705266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:40.705266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:40.705266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:40.705266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:40.705266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:40.705266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:40.705266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:40.705266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:40.705266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:40.705266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:40.711220 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:40.711220 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:40.711220 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:40.711220 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:40.711220 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 422/863] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:40.711220 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a84a11-274f-45b8-b70c-4d79942a4d55 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:40.730821 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 442/864] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Jun 27 18:27:40 2022] POST /identity/v3/services => generated 251 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:27:40.833829 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a84a11-274f-45b8-b70c-4d79942a4d55 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:40.880052 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f8db4-98be-482a-a712-3a2c5036e28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:40.880628 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f8db4-98be-482a-a712-3a2c5036e28d None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:40.880913 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f8db4-98be-482a-a712-3a2c5036e28d None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:40.881188 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f8db4-98be-482a-a712-3a2c5036e28d None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:40.881919 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f8db4-98be-482a-a712-3a2c5036e28d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:40.882228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f8db4-98be-482a-a712-3a2c5036e28d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:40.883704 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f8db4-98be-482a-a712-3a2c5036e28d None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:40.897916 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 423/865] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:40.903575 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778e698-2b41-4df9-8a28-e6f0af5c7d22 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:40.952543 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:41.009417 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778e698-2b41-4df9-8a28-e6f0af5c7d22 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:41.048503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a220edb9-da65-4938-9924-fac1809d0b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:41.049664 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220edb9-da65-4938-9924-fac1809d0b62 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:41.050171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220edb9-da65-4938-9924-fac1809d0b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:41.050693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a220edb9-da65-4938-9924-fac1809d0b62 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:41.059350 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a220edb9-da65-4938-9924-fac1809d0b62 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:41.061038 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a220edb9-da65-4938-9924-fac1809d0b62 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:41.076637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-a220edb9-da65-4938-9924-fac1809d0b62 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 18:27:41.076637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:41.076637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:41.076637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:41.076637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:41.076637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:41.076637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:41.076637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:41.076637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:41.076637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:41.076637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:41.076637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:41.076637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:41.076637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:41.076637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:41.076637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:41.077600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:41.077600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:41.077600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:41.077600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:41.077600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:41.077600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:41.077600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:41.077600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:41.077600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:41.077600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:41.077600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:41.077600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:41.077600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:41.077600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:41.077600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:41.077600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:41.077600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:41.077600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:41.078616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:41.078616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:41.078616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:41.078616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:41.078616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:41.078616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:41.078616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:41.078616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:41.078616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 18:27:41.078616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:41.081198 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 443/866] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Jun 27 18:27:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:27:41.087028 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f8db4-98be-482a-a712-3a2c5036e28d None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:41.118091 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:41.205828 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f8db4-98be-482a-a712-3a2c5036e28d None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:41.242744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187bb89d-aa2f-481e-a676-cedd343b6223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:41.243648 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187bb89d-aa2f-481e-a676-cedd343b6223 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:41.244142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187bb89d-aa2f-481e-a676-cedd343b6223 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:41.244565 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187bb89d-aa2f-481e-a676-cedd343b6223 None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:41.245551 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187bb89d-aa2f-481e-a676-cedd343b6223 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:41.246014 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187bb89d-aa2f-481e-a676-cedd343b6223 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:41.247654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187bb89d-aa2f-481e-a676-cedd343b6223 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:41.266179 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 424/867] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Jun 27 18:27:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:41.274480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a220edb9-da65-4938-9924-fac1809d0b62 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:41.376068 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a220edb9-da65-4938-9924-fac1809d0b62 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:41.383138 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:41.414452 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29453f-4efc-4530-a3f2-ebc7a7b23e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:41.415104 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29453f-4efc-4530-a3f2-ebc7a7b23e82 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:41.415394 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29453f-4efc-4530-a3f2-ebc7a7b23e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:41.415672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29453f-4efc-4530-a3f2-ebc7a7b23e82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:41.490626 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29453f-4efc-4530-a3f2-ebc7a7b23e82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:41.492141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29453f-4efc-4530-a3f2-ebc7a7b23e82 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:41.595153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 444/868] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:27:41 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 18:27:41.600650 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187bb89d-aa2f-481e-a676-cedd343b6223 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:41.692317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:41.702121 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187bb89d-aa2f-481e-a676-cedd343b6223 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:41.740853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e78c7-3ce9-4e23-b8c0-fbfe14762228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:41.747149 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e78c7-3ce9-4e23-b8c0-fbfe14762228 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:41.747442 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e78c7-3ce9-4e23-b8c0-fbfe14762228 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:41.747726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e78c7-3ce9-4e23-b8c0-fbfe14762228 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:41.758414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e78c7-3ce9-4e23-b8c0-fbfe14762228 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:41.760152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e78c7-3ce9-4e23-b8c0-fbfe14762228 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:41.778191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-a55e78c7-3ce9-4e23-b8c0-fbfe14762228 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:41.778191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:41.778191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:41.778191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:41.778191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:41.778191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:41.778191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:41.778191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:41.778191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:41.778191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:41.778191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:41.778191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:41.778191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:41.778191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:41.778191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:41.779129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:41.779129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:41.779129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:41.779129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:41.779129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:41.779129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:41.779129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:41.779129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:41.779129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:41.779129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:41.779129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:41.779129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:41.779129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:41.779129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:41.779129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:41.779129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:41.779129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:41.780031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:41.780031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:41.780031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:41.780031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:41.780031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 425/869] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:41.785223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29453f-4efc-4530-a3f2-ebc7a7b23e82 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:41.807268 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:41.825578 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1782909-6f23-4ad3-8a18-d2cf60808d7c None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:41.825977 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1782909-6f23-4ad3-8a18-d2cf60808d7c None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:41.826128 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1782909-6f23-4ad3-8a18-d2cf60808d7c None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:41.828423 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 426/870] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:41.839632 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc258db-7014-4a11-9c43-e161a3a6891f None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:41.839970 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc258db-7014-4a11-9c43-e161a3a6891f None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:41.840270 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc258db-7014-4a11-9c43-e161a3a6891f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:41.862340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-6fc258db-7014-4a11-9c43-e161a3a6891f None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:41.863765 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-6fc258db-7014-4a11-9c43-e161a3a6891f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:41.893629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29453f-4efc-4530-a3f2-ebc7a7b23e82 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:41.928541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ae5b4-93b5-4e8b-9b7b-4d1d4462f527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:41.929205 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ae5b4-93b5-4e8b-9b7b-4d1d4462f527 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:41.929507 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ae5b4-93b5-4e8b-9b7b-4d1d4462f527 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:41.929839 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ae5b4-93b5-4e8b-9b7b-4d1d4462f527 None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:41.930509 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ae5b4-93b5-4e8b-9b7b-4d1d4462f527 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:41.930832 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ae5b4-93b5-4e8b-9b7b-4d1d4462f527 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:41.932816 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ae5b4-93b5-4e8b-9b7b-4d1d4462f527 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:41.944721 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 427/871] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:41 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:41.952412 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 445/872] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:41.952561 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc258db-7014-4a11-9c43-e161a3a6891f None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:41.968143 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ae5b4-93b5-4e8b-9b7b-4d1d4462f527 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:42.040713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:42.048553 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:42.077385 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc258db-7014-4a11-9c43-e161a3a6891f None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:42.087630 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ae5b4-93b5-4e8b-9b7b-4d1d4462f527 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:42.118078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708196b-31c8-4dc0-a274-228cec5e57d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:42.119757 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708196b-31c8-4dc0-a274-228cec5e57d6 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:42.120653 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708196b-31c8-4dc0-a274-228cec5e57d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:42.121147 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708196b-31c8-4dc0-a274-228cec5e57d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:42.126315 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71845e72-7167-4cb7-95fc-c1c1709e273f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:42.127184 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71845e72-7167-4cb7-95fc-c1c1709e273f None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:42.127469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71845e72-7167-4cb7-95fc-c1c1709e273f None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:42.127819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71845e72-7167-4cb7-95fc-c1c1709e273f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:42.128493 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71845e72-7167-4cb7-95fc-c1c1709e273f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:42.130402 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71845e72-7167-4cb7-95fc-c1c1709e273f None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:42.130969 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708196b-31c8-4dc0-a274-228cec5e57d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:42.132767 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708196b-31c8-4dc0-a274-228cec5e57d6 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:42.143459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-71845e72-7167-4cb7-95fc-c1c1709e273f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 18:27:42.143459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:42.143459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:27:42.143459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:27:42.143459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:27:42.143459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:27:42.143459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:27:42.143459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:42.143459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 18:27:42.143459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:42.143459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:42.143459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:42.143459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:42.143459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:42.143459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:42.143459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:42.143459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:42.144629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:42.144629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:42.144629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:42.144629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:42.144629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:42.144629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:42.144629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:27:42.144629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:27:42.144629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:27:42.144629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:27:42.144629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:42.144629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:42.144629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:42.144629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:42.144629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:42.144629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:42.150646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:42.150646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:42.150646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:42.150646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:42.150646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:42.150646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:42.150646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:42.150646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:42.150646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:42.150646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:42.150646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:27:42.150646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:42.150646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 18:27:42.150646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:42.150646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 446/873] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Jun 27 18:27:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:42.151604 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b09f5-3fbb-4ac4-8d53-ab651c9d9966 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:42.151604 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b09f5-3fbb-4ac4-8d53-ab651c9d9966 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:42.151604 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b09f5-3fbb-4ac4-8d53-ab651c9d9966 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:42.151604 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 447/874] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:42.151604 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b09f5-3fbb-4ac4-8d53-ab651c9d9966 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:42.152482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-d708196b-31c8-4dc0-a274-228cec5e57d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:42.152482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:42.152482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:42.152482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:42.152482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:42.152482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:42.152482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:42.152482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:42.152482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:42.152482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:42.152482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:42.152482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:42.152482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:42.152482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:42.152482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:42.153325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:42.153325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:42.153325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:42.153325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:42.153325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:42.153325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:42.153325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:42.153325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:42.153325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:42.153325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:42.153325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:42.153325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:42.153325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:42.153325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:42.153325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:42.153325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:42.153325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:42.154208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:42.154208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:42.154208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:42.154208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:42.154208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:42.154208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:42.154208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:42.154208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:42.157145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 428/875] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:42.161871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8baf2-e44e-432a-890a-191d5551aed8 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:42.161871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8baf2-e44e-432a-890a-191d5551aed8 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:42.161871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8baf2-e44e-432a-890a-191d5551aed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:42.178812 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-dbd8baf2-e44e-432a-890a-191d5551aed8 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:42.179997 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-dbd8baf2-e44e-432a-890a-191d5551aed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:42.253581 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 429/876] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:42 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:42.254441 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd8baf2-e44e-432a-890a-191d5551aed8 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:42.264377 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b09f5-3fbb-4ac4-8d53-ab651c9d9966 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:42.294078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:42.300273 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6161d-b19e-464b-a62d-56410467e642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:42.304132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6161d-b19e-464b-a62d-56410467e642 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:42.304132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6161d-b19e-464b-a62d-56410467e642 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:42.304132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6161d-b19e-464b-a62d-56410467e642 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:42.304132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6161d-b19e-464b-a62d-56410467e642 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:42.304132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6161d-b19e-464b-a62d-56410467e642 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:42.304132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6161d-b19e-464b-a62d-56410467e642 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:42.309317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 448/877] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Jun 27 18:27:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:42.310763 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6161d-b19e-464b-a62d-56410467e642 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:42.327490 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:42.373234 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd8baf2-e44e-432a-890a-191d5551aed8 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:42.411694 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1ec7b-78d8-4175-9be8-bac113e33fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:42.412347 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1ec7b-78d8-4175-9be8-bac113e33fdc None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:42.412624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1ec7b-78d8-4175-9be8-bac113e33fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:42.412893 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1ec7b-78d8-4175-9be8-bac113e33fdc None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:42.413675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede1ec7b-78d8-4175-9be8-bac113e33fdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:42.413982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede1ec7b-78d8-4175-9be8-bac113e33fdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:42.415443 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede1ec7b-78d8-4175-9be8-bac113e33fdc None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:42.424618 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6161d-b19e-464b-a62d-56410467e642 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:42.429115 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 430/878] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:42.430093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1ec7b-78d8-4175-9be8-bac113e33fdc None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:42.462050 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c522c4-ef43-496a-8a4c-ee9c7761929e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:42.463273 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c522c4-ef43-496a-8a4c-ee9c7761929e None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:42.463691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c522c4-ef43-496a-8a4c-ee9c7761929e None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:42.463864 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c522c4-ef43-496a-8a4c-ee9c7761929e None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:42.464546 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c522c4-ef43-496a-8a4c-ee9c7761929e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:42.465917 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c522c4-ef43-496a-8a4c-ee9c7761929e None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:42.550515 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 449/879] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Jun 27 18:27:42 2022] POST /identity/v3/services => generated 269 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:27:42.550515 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c522c4-ef43-496a-8a4c-ee9c7761929e None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:42.561128 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1ec7b-78d8-4175-9be8-bac113e33fdc None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:42.583835 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:42.624003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e4f69-9aae-404b-9d2d-649c7aab0b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:42.625145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e4f69-9aae-404b-9d2d-649c7aab0b94 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:42.625429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e4f69-9aae-404b-9d2d-649c7aab0b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:42.625996 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e4f69-9aae-404b-9d2d-649c7aab0b94 None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:42.626758 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e4f69-9aae-404b-9d2d-649c7aab0b94 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:42.627064 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e4f69-9aae-404b-9d2d-649c7aab0b94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:42.628970 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e4f69-9aae-404b-9d2d-649c7aab0b94 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:42.634292 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:42.638462 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 431/880] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Jun 27 18:27:42 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:42.701705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c522c4-ef43-496a-8a4c-ee9c7761929e None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:42.747212 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358a3c5-79bd-4b9d-83b9-30151f957a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:42.747992 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358a3c5-79bd-4b9d-83b9-30151f957a7a None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:42.748278 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358a3c5-79bd-4b9d-83b9-30151f957a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:42.748554 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358a3c5-79bd-4b9d-83b9-30151f957a7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:42.749407 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e358a3c5-79bd-4b9d-83b9-30151f957a7a None admin] RBAC: Adding query filter params (role.id=c4bc6e72b9d647c5b2485d52d55f7dd4, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=58e1f1ac6ea14da0be6e9a24d007847a) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:42.749718 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e358a3c5-79bd-4b9d-83b9-30151f957a7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:42.751220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e358a3c5-79bd-4b9d-83b9-30151f957a7a None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:42.759383 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 450/881] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:27:42 2022] GET /identity/v3/role_assignments?user.id=58e1f1ac6ea14da0be6e9a24d007847a&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=c4bc6e72b9d647c5b2485d52d55f7dd4 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:42.980393 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa83017-0715-40a8-b990-6346742bd47d None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:42.980789 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa83017-0715-40a8-b990-6346742bd47d None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:42.980976 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa83017-0715-40a8-b990-6346742bd47d None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:42.982773 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 432/882] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:42.988119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad13030-7db0-493d-9d4c-af8f415e80ad None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:42.988698 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad13030-7db0-493d-9d4c-af8f415e80ad None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:42.989184 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad13030-7db0-493d-9d4c-af8f415e80ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:43.008066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-2ad13030-7db0-493d-9d4c-af8f415e80ad None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:43.010246 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-2ad13030-7db0-493d-9d4c-af8f415e80ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:43.081421 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 451/883] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:42 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:43.104124 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa83017-0715-40a8-b990-6346742bd47d None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:43.213960 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa83017-0715-40a8-b990-6346742bd47d None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:43.251043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac55e37-f66c-4bf1-b5ac-9a36750a20d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:43.252181 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac55e37-f66c-4bf1-b5ac-9a36750a20d6 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:43.252696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac55e37-f66c-4bf1-b5ac-9a36750a20d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:43.253213 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac55e37-f66c-4bf1-b5ac-9a36750a20d6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:43.254119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac55e37-f66c-4bf1-b5ac-9a36750a20d6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:43.255692 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac55e37-f66c-4bf1-b5ac-9a36750a20d6 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:43.268710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-9ac55e37-f66c-4bf1-b5ac-9a36750a20d6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 18:27:43.268710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:43.268710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:27:43.268710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:27:43.268710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:27:43.268710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:27:43.268710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:27:43.268710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:43.268710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 18:27:43.268710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:43.268710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:43.268710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:43.268710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:43.268710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:43.268710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:43.268710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:43.268710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:43.269764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:43.269764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:43.269764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:43.269764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:43.269764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:43.269764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:43.269764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:27:43.269764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:27:43.269764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:27:43.269764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:27:43.269764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:43.269764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:43.269764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:43.269764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:43.269764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:43.269764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:43.270624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:43.270624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:43.270624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:43.270624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:43.270624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:43.270624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:43.270624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:43.270624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:43.270624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:43.270624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:43.270624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:27:43.270624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:43.270624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 18:27:43.270624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:43.273479 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 433/884] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Jun 27 18:27:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:43.279584 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad13030-7db0-493d-9d4c-af8f415e80ad None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:43.332244 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:43.386006 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad13030-7db0-493d-9d4c-af8f415e80ad None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:43.425342 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace1414-2c8f-4b51-a01a-899a46fe8a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:43.426013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace1414-2c8f-4b51-a01a-899a46fe8a00 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:43.426262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace1414-2c8f-4b51-a01a-899a46fe8a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:43.426567 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace1414-2c8f-4b51-a01a-899a46fe8a00 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:43.427281 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace1414-2c8f-4b51-a01a-899a46fe8a00 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:43.427583 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace1414-2c8f-4b51-a01a-899a46fe8a00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:43.428938 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace1414-2c8f-4b51-a01a-899a46fe8a00 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:43.434889 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 452/885] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Jun 27 18:27:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:43.443498 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac55e37-f66c-4bf1-b5ac-9a36750a20d6 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:43.493349 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:43.552012 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac55e37-f66c-4bf1-b5ac-9a36750a20d6 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:43.585996 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6c55a-7563-48e6-9fe8-af8aaea29f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:43.586670 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6c55a-7563-48e6-9fe8-af8aaea29f5f None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:43.586948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6c55a-7563-48e6-9fe8-af8aaea29f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:43.587217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6c55a-7563-48e6-9fe8-af8aaea29f5f None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:43.588015 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6c55a-7563-48e6-9fe8-af8aaea29f5f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:43.588315 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6c55a-7563-48e6-9fe8-af8aaea29f5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:43.589663 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6c55a-7563-48e6-9fe8-af8aaea29f5f None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:43.595044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 434/886] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Jun 27 18:27:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:43.604061 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace1414-2c8f-4b51-a01a-899a46fe8a00 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:43.708786 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace1414-2c8f-4b51-a01a-899a46fe8a00 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:43.722806 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:43.757482 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37674d0-d0bb-4a89-90f6-9b6227e91b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:43.758435 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37674d0-d0bb-4a89-90f6-9b6227e91b0f None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:43.758936 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37674d0-d0bb-4a89-90f6-9b6227e91b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:43.759498 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37674d0-d0bb-4a89-90f6-9b6227e91b0f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:43.760513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37674d0-d0bb-4a89-90f6-9b6227e91b0f None admin] RBAC: Adding query filter params (interface=public, service_id=4b9e3298974b4792b19921d98a9ff7ca, region_id=RegionOne) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:43.761029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37674d0-d0bb-4a89-90f6-9b6227e91b0f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:43.762717 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37674d0-d0bb-4a89-90f6-9b6227e91b0f None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:43.768767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 453/887] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Jun 27 18:27:43 2022] GET /identity/v3/endpoints?service_id=4b9e3298974b4792b19921d98a9ff7ca&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:43.777197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6c55a-7563-48e6-9fe8-af8aaea29f5f None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:43.883716 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6c55a-7563-48e6-9fe8-af8aaea29f5f None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:43.891820 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:43.919388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb598f79-e64f-4c44-a90f-55df98800485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:43.920393 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb598f79-e64f-4c44-a90f-55df98800485 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:43.920907 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb598f79-e64f-4c44-a90f-55df98800485 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:43.921382 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb598f79-e64f-4c44-a90f-55df98800485 None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:43.922269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb598f79-e64f-4c44-a90f-55df98800485 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:43.923873 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb598f79-e64f-4c44-a90f-55df98800485 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:43.930217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 435/888] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Jun 27 18:27:43 2022] GET /identity/v3/services => generated 1268 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:27:44.721691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2970657-e278-4f17-a9f2-2b112b468044 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:44.722213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2970657-e278-4f17-a9f2-2b112b468044 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:44.722213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2970657-e278-4f17-a9f2-2b112b468044 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:44.723894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 454/889] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:44.729104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e9f58-1025-4368-a203-2217a6533c28 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:44.729935 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e9f58-1025-4368-a203-2217a6533c28 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:44.730477 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e9f58-1025-4368-a203-2217a6533c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:44.753759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-7b9e9f58-1025-4368-a203-2217a6533c28 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:44.755215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-7b9e9f58-1025-4368-a203-2217a6533c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:44.786476 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7597a-3c1a-4d2f-be34-c77e9063097f None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:44.787117 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7597a-3c1a-4d2f-be34-c77e9063097f None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:44.787626 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7597a-3c1a-4d2f-be34-c77e9063097f None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:44.789458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 455/890] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:27:44.795478 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0dcdd-b6f6-4392-8aec-d57213cac774 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:44.796018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0dcdd-b6f6-4392-8aec-d57213cac774 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:44.796529 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0dcdd-b6f6-4392-8aec-d57213cac774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:44.813866 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-37d0dcdd-b6f6-4392-8aec-d57213cac774 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:44.815453 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-37d0dcdd-b6f6-4392-8aec-d57213cac774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:44.824219 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 436/891] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:44.847306 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e9f58-1025-4368-a203-2217a6533c28 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:44.891046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 456/892] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 18:27:44.916362 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0dcdd-b6f6-4392-8aec-d57213cac774 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:44.963970 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:44.967434 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e9f58-1025-4368-a203-2217a6533c28 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:45.006045 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e8793-9f41-4c27-8ead-5cfe66b3f6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:45.006907 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e8793-9f41-4c27-8ead-5cfe66b3f6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:45.007231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e8793-9f41-4c27-8ead-5cfe66b3f6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:45.007509 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e8793-9f41-4c27-8ead-5cfe66b3f6d6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:45.008168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e8793-9f41-4c27-8ead-5cfe66b3f6d6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:45.009553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e8793-9f41-4c27-8ead-5cfe66b3f6d6 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:45.022125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-bf1e8793-9f41-4c27-8ead-5cfe66b3f6d6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 18:27:45.022125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:45.022125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:27:45.022125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:27:45.022125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:27:45.022125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:27:45.022125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:27:45.022125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:45.022125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 18:27:45.022125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:45.022125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:45.022125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:45.022125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:45.022125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:45.022125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:45.022125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:45.023140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:45.023140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:45.023140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:45.023140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:45.023140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:45.023140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:45.023140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:45.023140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:27:45.023140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:27:45.023140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:27:45.023140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:27:45.023140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:45.023140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:45.023140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:45.023140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:45.023140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:45.023989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:45.023989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:45.023989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:45.023989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:45.023989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:45.023989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:45.023989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:45.023989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:45.023989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:45.023989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:45.023989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:45.023989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:27:45.023989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:45.023989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 18:27:45.023989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:45.023989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 437/893] 10.222.0.15 () {56 vars in 1132 bytes} [Mon Jun 27 18:27:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:27:45.024852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f910e-09fd-4aa0-9a56-7fd9bcf72ef1 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:45.024852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f910e-09fd-4aa0-9a56-7fd9bcf72ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:45.025948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f910e-09fd-4aa0-9a56-7fd9bcf72ef1 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:45.026748 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0dcdd-b6f6-4392-8aec-d57213cac774 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:45.027182 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 438/894] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:45.032274 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e32d0f-891d-4876-bc7e-b117c638e625 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:45.032559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e32d0f-891d-4876-bc7e-b117c638e625 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:45.032895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e32d0f-891d-4876-bc7e-b117c638e625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:45.039452 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:45.050315 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-46e32d0f-891d-4876-bc7e-b117c638e625 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:45.051575 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-46e32d0f-891d-4876-bc7e-b117c638e625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:45.068486 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6d871-fa0d-480b-aba9-df84fe42f6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:45.069153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6d871-fa0d-480b-aba9-df84fe42f6bf None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:45.069438 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6d871-fa0d-480b-aba9-df84fe42f6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:45.069714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6d871-fa0d-480b-aba9-df84fe42f6bf None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:45.070467 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6d871-fa0d-480b-aba9-df84fe42f6bf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:45.072331 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6d871-fa0d-480b-aba9-df84fe42f6bf None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:45.130745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 439/895] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 18:27:45.130745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e32d0f-891d-4876-bc7e-b117c638e625 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:45.142900 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 457/896] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Jun 27 18:27:44 2022] POST /identity/v3/services => generated 241 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:27:45.147225 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6d871-fa0d-480b-aba9-df84fe42f6bf None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:45.253505 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:45.266233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e32d0f-891d-4876-bc7e-b117c638e625 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:45.297041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6d871-fa0d-480b-aba9-df84fe42f6bf None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:45.318050 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a59d3-056c-4ef0-8fa4-cd1180dd08d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:45.323058 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a59d3-056c-4ef0-8fa4-cd1180dd08d8 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:45.323058 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a59d3-056c-4ef0-8fa4-cd1180dd08d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:45.323058 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a59d3-056c-4ef0-8fa4-cd1180dd08d8 None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:45.323058 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a59d3-056c-4ef0-8fa4-cd1180dd08d8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:45.323058 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a59d3-056c-4ef0-8fa4-cd1180dd08d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:45.323442 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a59d3-056c-4ef0-8fa4-cd1180dd08d8 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:45.329497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 440/897] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Jun 27 18:27:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:45.335528 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a59d3-056c-4ef0-8fa4-cd1180dd08d8 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:45.340953 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e98d11-499f-4f5f-a15a-970b4d72f9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:45.341769 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e98d11-499f-4f5f-a15a-970b4d72f9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:45.342055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e98d11-499f-4f5f-a15a-970b4d72f9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:45.342336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e98d11-499f-4f5f-a15a-970b4d72f9c6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:45.355641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e98d11-499f-4f5f-a15a-970b4d72f9c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:45.364403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-40e98d11-499f-4f5f-a15a-970b4d72f9c6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 18:27:45.364403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:45.364403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:45.364403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:45.364403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:45.364403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:45.364403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:45.364403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:45.364403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:45.364403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:45.364403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:45.364403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:45.364403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:45.364403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:45.364403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:45.364403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:45.365558 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:45.365558 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:45.365558 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:45.365558 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:45.365558 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:45.365558 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:45.365558 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:45.365558 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:45.365558 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:45.365558 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:45.365558 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:45.365558 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:45.365558 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:45.365558 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:45.365558 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:45.365558 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:45.367120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:45.367120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:45.367120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:45.367120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:45.367120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:45.367120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:45.367120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:45.367120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:45.367120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:45.367120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:45.367120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 18:27:45.367120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:45.367120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 458/898] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Jun 27 18:27:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:45.372439 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e98d11-499f-4f5f-a15a-970b4d72f9c6 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:45.405026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:45.458910 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a59d3-056c-4ef0-8fa4-cd1180dd08d8 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:45.503468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a013a-4257-4c4f-bd37-36ab6cbb1f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:45.504137 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a013a-4257-4c4f-bd37-36ab6cbb1f7f None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:45.504594 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a013a-4257-4c4f-bd37-36ab6cbb1f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:45.504878 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a013a-4257-4c4f-bd37-36ab6cbb1f7f None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:45.505837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a013a-4257-4c4f-bd37-36ab6cbb1f7f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:45.506147 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a013a-4257-4c4f-bd37-36ab6cbb1f7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:45.508270 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a013a-4257-4c4f-bd37-36ab6cbb1f7f None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:45.513992 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e98d11-499f-4f5f-a15a-970b4d72f9c6 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:45.518525 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 441/899] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Jun 27 18:27:45 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:45.520543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a013a-4257-4c4f-bd37-36ab6cbb1f7f None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:45.555158 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:45.565597 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c252e-090d-4d85-ab33-fe745798486a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:45.566576 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c252e-090d-4d85-ab33-fe745798486a None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:45.566865 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c252e-090d-4d85-ab33-fe745798486a None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:45.567143 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c252e-090d-4d85-ab33-fe745798486a None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:45.567902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c252e-090d-4d85-ab33-fe745798486a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:45.568211 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c252e-090d-4d85-ab33-fe745798486a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:45.569750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c252e-090d-4d85-ab33-fe745798486a None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:45.582155 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 459/900] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Jun 27 18:27:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:45.587411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c252e-090d-4d85-ab33-fe745798486a None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:45.656874 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a013a-4257-4c4f-bd37-36ab6cbb1f7f None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:45.702314 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:45.705047 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3985891-920a-4533-87d7-0992f0bac53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:45.705980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3985891-920a-4533-87d7-0992f0bac53f None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:45.706976 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3985891-920a-4533-87d7-0992f0bac53f None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:45.707459 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3985891-920a-4533-87d7-0992f0bac53f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:45.708749 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3985891-920a-4533-87d7-0992f0bac53f None admin] RBAC: Adding query filter params (interface=public, service_id=d44dbf56975548b89fcf78447ce9f181, region_id=RegionOne) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:45.709416 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3985891-920a-4533-87d7-0992f0bac53f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:45.711587 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3985891-920a-4533-87d7-0992f0bac53f None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:45.723765 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c252e-090d-4d85-ab33-fe745798486a None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:45.724285 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 442/901] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Jun 27 18:27:45 2022] GET /identity/v3/endpoints?service_id=d44dbf56975548b89fcf78447ce9f181&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:45.726166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3985891-920a-4533-87d7-0992f0bac53f None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:45.763319 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca962dc-183e-4d72-ab91-c0ad6e464d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:45.764515 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca962dc-183e-4d72-ab91-c0ad6e464d63 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:45.765013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca962dc-183e-4d72-ab91-c0ad6e464d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:45.765580 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca962dc-183e-4d72-ab91-c0ad6e464d63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:45.777784 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca962dc-183e-4d72-ab91-c0ad6e464d63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:45.779525 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca962dc-183e-4d72-ab91-c0ad6e464d63 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:45.797022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-fca962dc-183e-4d72-ab91-c0ad6e464d63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:45.797022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:45.797022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:45.797022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:45.797022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:45.797022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:45.797022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:45.797022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:45.797022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:45.797022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:45.797022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:45.797022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:45.797022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:45.797022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:45.797022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:45.797969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:45.797969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:45.797969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:45.797969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:45.797969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:45.797969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:45.797969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:45.797969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:45.797969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:45.797969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:45.797969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:45.797969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:45.797969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:45.797969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:45.797969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:45.797969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:45.797969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:45.798873 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:45.798873 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:45.798873 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:45.798873 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:45.801087 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 460/902] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:27:45.811501 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca962dc-183e-4d72-ab91-c0ad6e464d63 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:45.821601 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:45.838642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3985891-920a-4533-87d7-0992f0bac53f None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:45.877424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1264c-3f0c-4680-85db-738ff27d77f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:45.878194 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1264c-3f0c-4680-85db-738ff27d77f5 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:45.878510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1264c-3f0c-4680-85db-738ff27d77f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:45.878789 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1264c-3f0c-4680-85db-738ff27d77f5 None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:45.879594 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1264c-3f0c-4680-85db-738ff27d77f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:45.880968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1264c-3f0c-4680-85db-738ff27d77f5 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:45.888406 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 443/903] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Jun 27 18:27:45 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:27:45.921827 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca962dc-183e-4d72-ab91-c0ad6e464d63 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:45.960239 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315a15d-027a-4d2e-95f4-e2affea49e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:45.961272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315a15d-027a-4d2e-95f4-e2affea49e34 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:45.961719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315a15d-027a-4d2e-95f4-e2affea49e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:45.962144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315a15d-027a-4d2e-95f4-e2affea49e34 None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:45.963272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d315a15d-027a-4d2e-95f4-e2affea49e34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:45.963825 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d315a15d-027a-4d2e-95f4-e2affea49e34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:45.965374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d315a15d-027a-4d2e-95f4-e2affea49e34 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:45.968456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:45.980360 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 461/904] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:45.986605 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1264c-3f0c-4680-85db-738ff27d77f5 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:45.998080 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5224197-9f16-48f6-b178-0c51b8d84b2a None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:45.998439 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5224197-9f16-48f6-b178-0c51b8d84b2a None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:45.998667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5224197-9f16-48f6-b178-0c51b8d84b2a None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:46.000330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 462/905] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:46.006137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dc27a-161d-4f74-9fe5-4ed0867ea774 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:46.006463 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dc27a-161d-4f74-9fe5-4ed0867ea774 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:46.006727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73dc27a-161d-4f74-9fe5-4ed0867ea774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:46.026937 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-a73dc27a-161d-4f74-9fe5-4ed0867ea774 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:46.028661 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-a73dc27a-161d-4f74-9fe5-4ed0867ea774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:46.102698 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1264c-3f0c-4680-85db-738ff27d77f5 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:46.114137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 463/906] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:46.137535 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73dc27a-161d-4f74-9fe5-4ed0867ea774 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:46.145416 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39f5cd-2724-4d15-841c-e8b621ef484d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:46.146224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39f5cd-2724-4d15-841c-e8b621ef484d None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:46.146540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39f5cd-2724-4d15-841c-e8b621ef484d None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:46.146821 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39f5cd-2724-4d15-841c-e8b621ef484d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:46.154676 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca39f5cd-2724-4d15-841c-e8b621ef484d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:46.156183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca39f5cd-2724-4d15-841c-e8b621ef484d None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:46.172178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-ca39f5cd-2724-4d15-841c-e8b621ef484d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 18:27:46.172178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:46.172178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:46.172178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:46.172178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:46.172178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:46.172178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:46.172178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:46.172178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:46.172178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:46.172178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:46.172178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:46.172178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:46.172178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:46.172178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:46.172178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:46.173196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:46.173196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:46.173196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:46.173196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:46.173196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:46.173196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:46.173196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:46.173196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:46.173196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:46.173196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:46.173196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:46.173196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:46.173196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:46.173196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:46.173196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:46.173196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:46.173196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:46.173196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:46.174156 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:46.174156 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:46.174156 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:46.174156 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:46.174156 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:46.174156 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:46.174156 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:46.174156 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:46.174156 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 18:27:46.174156 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:46.174762 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 444/907] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Jun 27 18:27:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:46.186725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39f5cd-2724-4d15-841c-e8b621ef484d None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:46.202896 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:46.264495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73dc27a-161d-4f74-9fe5-4ed0867ea774 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:46.294073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:46.304211 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6a0f7-7651-458e-a2cb-0b222168d2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:46.305210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6a0f7-7651-458e-a2cb-0b222168d2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:46.305968 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6a0f7-7651-458e-a2cb-0b222168d2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:46.306227 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6a0f7-7651-458e-a2cb-0b222168d2e3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:46.307139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6a0f7-7651-458e-a2cb-0b222168d2e3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:46.308788 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6a0f7-7651-458e-a2cb-0b222168d2e3 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:46.319723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39f5cd-2724-4d15-841c-e8b621ef484d None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:46.322372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-e8f6a0f7-7651-458e-a2cb-0b222168d2e3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 18:27:46.322372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:46.322372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:27:46.322372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:27:46.322372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:27:46.322372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:27:46.322372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:27:46.322372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:46.322372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 18:27:46.322372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:46.322372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:46.322372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:46.322372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:46.322372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:46.322372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:46.322372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:46.322372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:46.323610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:46.323610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:46.323610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:46.323610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:46.323610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:46.323610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:46.323610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:27:46.323610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:27:46.323610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:27:46.323610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:27:46.323610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:46.323610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:46.323610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:46.323610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:46.323610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:46.323610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:46.324504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:46.324504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:46.324504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:46.324504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:46.324504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:46.324504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:46.324504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:46.324504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:46.324504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:46.324504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:46.324504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:27:46.324504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:46.324504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 18:27:46.324504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:46.332453 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 464/908] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Jun 27 18:27:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:27:46.333485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6a0f7-7651-458e-a2cb-0b222168d2e3 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:46.365040 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c703487-8db2-45fe-9a62-7ddddaeed021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:46.366028 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c703487-8db2-45fe-9a62-7ddddaeed021 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:46.366315 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c703487-8db2-45fe-9a62-7ddddaeed021 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:46.366676 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c703487-8db2-45fe-9a62-7ddddaeed021 None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:46.367839 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c703487-8db2-45fe-9a62-7ddddaeed021 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:46.368176 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c703487-8db2-45fe-9a62-7ddddaeed021 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:46.371118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c703487-8db2-45fe-9a62-7ddddaeed021 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:46.391232 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 445/909] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Jun 27 18:27:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:46.397670 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c703487-8db2-45fe-9a62-7ddddaeed021 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:46.453365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6a0f7-7651-458e-a2cb-0b222168d2e3 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:46.491742 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:46.497453 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1ee3c-471c-40c9-ab41-b136f08d797c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:46.498175 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1ee3c-471c-40c9-ab41-b136f08d797c None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:46.498501 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1ee3c-471c-40c9-ab41-b136f08d797c None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:46.498790 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1ee3c-471c-40c9-ab41-b136f08d797c None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:46.499561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1ee3c-471c-40c9-ab41-b136f08d797c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:46.499875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1ee3c-471c-40c9-ab41-b136f08d797c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:46.501496 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1ee3c-471c-40c9-ab41-b136f08d797c None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:46.512155 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 465/910] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Jun 27 18:27:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:46.513051 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1ee3c-471c-40c9-ab41-b136f08d797c None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:46.528972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c703487-8db2-45fe-9a62-7ddddaeed021 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:46.565270 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb32d4-1b47-428f-ad9f-b8da2d5fc0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:46.565932 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb32d4-1b47-428f-ad9f-b8da2d5fc0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:46.566285 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb32d4-1b47-428f-ad9f-b8da2d5fc0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:46.566652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb32d4-1b47-428f-ad9f-b8da2d5fc0d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:46.575011 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:46.625183 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1ee3c-471c-40c9-ab41-b136f08d797c None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:46.641101 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb32d4-1b47-428f-ad9f-b8da2d5fc0d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:46.642697 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb32d4-1b47-428f-ad9f-b8da2d5fc0d2 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:46.664242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c189c53-c6e5-470b-aa4a-8c8ac7770d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:46.665090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c189c53-c6e5-470b-aa4a-8c8ac7770d09 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:46.665381 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c189c53-c6e5-470b-aa4a-8c8ac7770d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:46.666190 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c189c53-c6e5-470b-aa4a-8c8ac7770d09 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:46.667270 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c189c53-c6e5-470b-aa4a-8c8ac7770d09 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:46.667690 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c189c53-c6e5-470b-aa4a-8c8ac7770d09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:46.669068 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c189c53-c6e5-470b-aa4a-8c8ac7770d09 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:46.679505 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 466/911] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Jun 27 18:27:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:46.680351 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c189c53-c6e5-470b-aa4a-8c8ac7770d09 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:46.709578 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:46.746353 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 446/912] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:27:46 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 18:27:46.756702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb32d4-1b47-428f-ad9f-b8da2d5fc0d2 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:46.799504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c189c53-c6e5-470b-aa4a-8c8ac7770d09 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:46.818977 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:46.837167 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cb29a-7d12-4fd6-b579-ca233f59ed5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:46.839794 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cb29a-7d12-4fd6-b579-ca233f59ed5d None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:46.839794 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cb29a-7d12-4fd6-b579-ca233f59ed5d None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:46.839794 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cb29a-7d12-4fd6-b579-ca233f59ed5d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:46.840612 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479cb29a-7d12-4fd6-b579-ca233f59ed5d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:46.842318 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479cb29a-7d12-4fd6-b579-ca233f59ed5d None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:46.860386 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb32d4-1b47-428f-ad9f-b8da2d5fc0d2 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:46.895814 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7c0d3-9dea-4265-8123-169d77f8e600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:46.896599 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7c0d3-9dea-4265-8123-169d77f8e600 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:46.896883 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7c0d3-9dea-4265-8123-169d77f8e600 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:46.897159 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7c0d3-9dea-4265-8123-169d77f8e600 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:46.908024 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a7c0d3-9dea-4265-8123-169d77f8e600 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:46.909617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a7c0d3-9dea-4265-8123-169d77f8e600 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:46.928323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-21a7c0d3-9dea-4265-8123-169d77f8e600 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:46.928323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:46.928323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:46.928323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:46.928323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:46.928323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:46.928323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:46.928323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:46.928323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:46.928323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:46.928323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:46.928323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:46.928323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:46.928323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:46.928323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:46.929335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:46.929335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:46.929335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:46.929335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:46.929335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:46.929335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:46.929335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:46.929335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:46.929335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:46.929335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:46.929335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:46.929335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:46.929335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:46.929335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:46.929335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:46.929335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:46.929335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:46.940623 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:46.940623 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:46.940623 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:46.940623 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:46.940623 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 447/913] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:27:46.940915 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 467/914] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Jun 27 18:27:46 2022] POST /identity/v3/endpoints => generated 346 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:27:46.940991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7c0d3-9dea-4265-8123-169d77f8e600 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:46.966733 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:47.057942 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7c0d3-9dea-4265-8123-169d77f8e600 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:47.106872 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a07fd3-7305-46c1-9351-837c06ca7f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:47.107881 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a07fd3-7305-46c1-9351-837c06ca7f81 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:47.108374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a07fd3-7305-46c1-9351-837c06ca7f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:47.108835 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a07fd3-7305-46c1-9351-837c06ca7f81 None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:47.109733 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a07fd3-7305-46c1-9351-837c06ca7f81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:47.110231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a07fd3-7305-46c1-9351-837c06ca7f81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:47.111810 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a07fd3-7305-46c1-9351-837c06ca7f81 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:47.125221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 448/915] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:47.131310 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cb29a-7d12-4fd6-b579-ca233f59ed5d None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:47.239999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cb29a-7d12-4fd6-b579-ca233f59ed5d None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:47.277014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53f99a-434a-4b4e-9b20-18ec436c417d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:47.278160 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53f99a-434a-4b4e-9b20-18ec436c417d None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:47.280771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53f99a-434a-4b4e-9b20-18ec436c417d None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:47.280771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53f99a-434a-4b4e-9b20-18ec436c417d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:47.286496 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53f99a-434a-4b4e-9b20-18ec436c417d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:47.287951 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53f99a-434a-4b4e-9b20-18ec436c417d None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:47.302568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-4d53f99a-434a-4b4e-9b20-18ec436c417d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:47.302568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:47.302568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:47.302568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:47.302568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:47.302568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:47.302568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:47.302568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:47.302568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:47.302568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:47.302568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:47.302568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:47.302568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:47.302568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:47.302568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:47.303508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:47.303508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:47.303508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:47.303508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:47.303508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:47.303508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:47.303508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:47.303508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:47.303508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:47.303508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:47.303508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:47.303508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:47.303508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:47.303508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:47.303508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:47.303508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:47.303508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:47.304416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:47.304416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:47.304416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:47.304416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:47.304416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:47.304416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:47.304416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:47.304416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:47.304416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 468/916] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:47.311744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a07fd3-7305-46c1-9351-837c06ca7f81 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:47.362505 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51540802-e714-4d51-8b84-669530784b67 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:47.362696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51540802-e714-4d51-8b84-669530784b67 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:47.362980 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51540802-e714-4d51-8b84-669530784b67 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:47.364478 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 469/917] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:27:47.369893 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4547d8-f407-44cd-b89d-1f7544ffe8ea None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:47.370180 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4547d8-f407-44cd-b89d-1f7544ffe8ea None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:47.370500 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4547d8-f407-44cd-b89d-1f7544ffe8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:47.391718 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-2f4547d8-f407-44cd-b89d-1f7544ffe8ea None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:47.393428 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-2f4547d8-f407-44cd-b89d-1f7544ffe8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:47.404363 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:47.430323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a07fd3-7305-46c1-9351-837c06ca7f81 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:47.442768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:47.470655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ab04d-4f28-48fa-8fb8-1112dd613c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:47.471730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ab04d-4f28-48fa-8fb8-1112dd613c9f None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:47.472216 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ab04d-4f28-48fa-8fb8-1112dd613c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:47.472551 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 470/918] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:47.472961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ab04d-4f28-48fa-8fb8-1112dd613c9f None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:47.473917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ab04d-4f28-48fa-8fb8-1112dd613c9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:47.474229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ab04d-4f28-48fa-8fb8-1112dd613c9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:47.475708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ab04d-4f28-48fa-8fb8-1112dd613c9f None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:47.491113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 449/919] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:47.498134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4547d8-f407-44cd-b89d-1f7544ffe8ea None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:47.498337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ab04d-4f28-48fa-8fb8-1112dd613c9f None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:47.606793 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4547d8-f407-44cd-b89d-1f7544ffe8ea None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:47.620292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ab04d-4f28-48fa-8fb8-1112dd613c9f None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:47.643246 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838ad36-3bf2-43d5-a66e-fb6ba889cd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:47.644005 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838ad36-3bf2-43d5-a66e-fb6ba889cd57 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:47.644403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838ad36-3bf2-43d5-a66e-fb6ba889cd57 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:47.644809 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838ad36-3bf2-43d5-a66e-fb6ba889cd57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:47.645752 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838ad36-3bf2-43d5-a66e-fb6ba889cd57 None admin] RBAC: Adding query filter params (role.id=3ef93c53540740d6b89875c0ff507fab, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=58e1f1ac6ea14da0be6e9a24d007847a) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:47.646176 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838ad36-3bf2-43d5-a66e-fb6ba889cd57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:47.647749 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838ad36-3bf2-43d5-a66e-fb6ba889cd57 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:47.653874 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 471/920] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:27:47 2022] GET /identity/v3/role_assignments?user.id=58e1f1ac6ea14da0be6e9a24d007847a&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=3ef93c53540740d6b89875c0ff507fab => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:47.660104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d7c94-84a6-49a6-9ca7-60359da40f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:47.660940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d7c94-84a6-49a6-9ca7-60359da40f8a None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:47.661221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d7c94-84a6-49a6-9ca7-60359da40f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:47.661491 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d7c94-84a6-49a6-9ca7-60359da40f8a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:47.662160 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d7c94-84a6-49a6-9ca7-60359da40f8a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:47.663710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d7c94-84a6-49a6-9ca7-60359da40f8a None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:47.667771 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:47.678677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-ee3d7c94-84a6-49a6-9ca7-60359da40f8a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 18:27:47.678677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:47.678677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:27:47.678677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:27:47.678677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:27:47.678677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:27:47.678677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:27:47.678677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:47.678677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 18:27:47.678677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:47.678677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:47.678677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:47.678677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:47.678677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:47.678677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:47.678677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:47.678677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:47.686240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:47.686240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:47.686240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:47.686240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:47.686240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:47.686240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:47.686240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:27:47.686240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:27:47.686240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:27:47.686240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:27:47.686240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:47.686240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:47.686240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:47.686240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:47.686240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:47.686240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:47.687140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:47.687140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:47.687140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:47.687140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:47.687140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:47.687140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:47.687140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:47.687140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:47.687140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:47.687140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:47.687140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:27:47.687140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:47.687140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 18:27:47.687140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:47.687140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 450/921] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Jun 27 18:27:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:47.687946 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838ad36-3bf2-43d5-a66e-fb6ba889cd57 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:47.719846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:47.813014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838ad36-3bf2-43d5-a66e-fb6ba889cd57 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:47.848904 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3f8e0-d1c6-4af5-aa75-f9d6430f6314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:47.849623 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3f8e0-d1c6-4af5-aa75-f9d6430f6314 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:47.849904 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3f8e0-d1c6-4af5-aa75-f9d6430f6314 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:47.850178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3f8e0-d1c6-4af5-aa75-f9d6430f6314 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:47.850965 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a3f8e0-d1c6-4af5-aa75-f9d6430f6314 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:47.851270 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a3f8e0-d1c6-4af5-aa75-f9d6430f6314 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:47.852642 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a3f8e0-d1c6-4af5-aa75-f9d6430f6314 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:47.862496 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 472/922] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Jun 27 18:27:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:47.864358 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d7c94-84a6-49a6-9ca7-60359da40f8a None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:47.971012 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d7c94-84a6-49a6-9ca7-60359da40f8a None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:48.006739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338d06e-1390-4678-aaf5-03efb35e06aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:48.007395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338d06e-1390-4678-aaf5-03efb35e06aa None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:48.007676 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338d06e-1390-4678-aaf5-03efb35e06aa None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:48.007953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338d06e-1390-4678-aaf5-03efb35e06aa None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:48.008661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338d06e-1390-4678-aaf5-03efb35e06aa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:48.008969 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338d06e-1390-4678-aaf5-03efb35e06aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:48.010479 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338d06e-1390-4678-aaf5-03efb35e06aa None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:48.019458 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 451/923] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Jun 27 18:27:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:48.022139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3f8e0-d1c6-4af5-aa75-f9d6430f6314 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:48.121431 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:48.130231 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3f8e0-d1c6-4af5-aa75-f9d6430f6314 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:48.139641 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e7fde-641d-4342-abb8-1ea7b988ecf6 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:48.139928 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e7fde-641d-4342-abb8-1ea7b988ecf6 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:48.140209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e7fde-641d-4342-abb8-1ea7b988ecf6 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:48.142798 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 452/924] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:27:48.146977 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd5ae6-0f5b-4242-8bb3-8f96ba3366fe None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:48.147272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd5ae6-0f5b-4242-8bb3-8f96ba3366fe None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:48.157254 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd5ae6-0f5b-4242-8bb3-8f96ba3366fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:48.166625 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-19dd5ae6-0f5b-4242-8bb3-8f96ba3366fe None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:48.167539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f5f53-b79b-4921-8a84-372a1abee561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:48.168048 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-19dd5ae6-0f5b-4242-8bb3-8f96ba3366fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:48.168238 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f5f53-b79b-4921-8a84-372a1abee561 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:48.168495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f5f53-b79b-4921-8a84-372a1abee561 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:48.168776 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f5f53-b79b-4921-8a84-372a1abee561 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:48.169604 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f5f53-b79b-4921-8a84-372a1abee561 None admin] RBAC: Adding query filter params (interface=public, service_id=b48ac84e610b4bd2adf2be27253358d6, region_id=RegionOne) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:48.169918 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f5f53-b79b-4921-8a84-372a1abee561 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:48.171370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f5f53-b79b-4921-8a84-372a1abee561 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:48.178042 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 473/925] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Jun 27 18:27:48 2022] GET /identity/v3/endpoints?service_id=b48ac84e610b4bd2adf2be27253358d6&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:48.187357 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f5f53-b79b-4921-8a84-372a1abee561 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:48.251510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:48.262484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 453/926] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:48.285357 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd5ae6-0f5b-4242-8bb3-8f96ba3366fe None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:48.316611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f5f53-b79b-4921-8a84-372a1abee561 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:48.360939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9af89-81a3-4aa5-9c00-103dda791d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:48.362147 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9af89-81a3-4aa5-9c00-103dda791d7d None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:48.362955 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9af89-81a3-4aa5-9c00-103dda791d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:48.363828 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9af89-81a3-4aa5-9c00-103dda791d7d None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:48.364724 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9af89-81a3-4aa5-9c00-103dda791d7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:48.366487 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9af89-81a3-4aa5-9c00-103dda791d7d None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:48.383051 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 474/927] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Jun 27 18:27:48 2022] GET /identity/v3/services => generated 1497 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:27:48.412271 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd5ae6-0f5b-4242-8bb3-8f96ba3366fe None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:48.452885 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfebe9-50c2-4a5a-bbb5-18179db2b599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:48.453695 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfebe9-50c2-4a5a-bbb5-18179db2b599 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:48.453973 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfebe9-50c2-4a5a-bbb5-18179db2b599 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:48.454241 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfebe9-50c2-4a5a-bbb5-18179db2b599 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:48.454940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdfebe9-50c2-4a5a-bbb5-18179db2b599 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:48.456466 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdfebe9-50c2-4a5a-bbb5-18179db2b599 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:48.470696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-dfdfebe9-50c2-4a5a-bbb5-18179db2b599 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 18:27:48.470696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:48.470696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:27:48.470696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:27:48.470696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:27:48.470696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:27:48.470696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:27:48.470696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:48.470696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 18:27:48.470696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:48.470696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:48.470696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:48.470696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:48.470696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:48.470696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:48.470696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:48.471702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:48.471702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:48.471702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:48.471702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:48.471702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:48.471702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:48.471702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:48.471702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:27:48.471702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:27:48.471702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:27:48.471702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:27:48.471702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:48.471702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:48.471702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:48.471702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:48.471702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:48.472975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:48.472975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:48.472975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:48.472975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:48.472975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:48.472975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:48.472975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:48.472975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:48.472975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:48.472975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:48.472975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:48.472975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:27:48.472975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:48.472975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 18:27:48.472975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:48.472975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 454/928] 10.222.0.15 () {56 vars in 1132 bytes} [Mon Jun 27 18:27:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:27:48.480760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9af89-81a3-4aa5-9c00-103dda791d7d None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:48.505508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:48.602509 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9af89-81a3-4aa5-9c00-103dda791d7d None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:48.639454 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904c8cd-2c9f-47c8-947d-de0b134c38b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:48.640119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904c8cd-2c9f-47c8-947d-de0b134c38b5 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:48.640407 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904c8cd-2c9f-47c8-947d-de0b134c38b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:48.640691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904c8cd-2c9f-47c8-947d-de0b134c38b5 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:48.641417 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6904c8cd-2c9f-47c8-947d-de0b134c38b5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:48.641727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6904c8cd-2c9f-47c8-947d-de0b134c38b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:48.643144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6904c8cd-2c9f-47c8-947d-de0b134c38b5 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:48.648493 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 475/929] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Jun 27 18:27:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:48.654637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfebe9-50c2-4a5a-bbb5-18179db2b599 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:48.718794 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:48.778714 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfebe9-50c2-4a5a-bbb5-18179db2b599 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:48.823572 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205848c8-8dbc-47bd-93e4-a886dddf7d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:48.824609 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205848c8-8dbc-47bd-93e4-a886dddf7d6c None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:48.825109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205848c8-8dbc-47bd-93e4-a886dddf7d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:48.825590 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205848c8-8dbc-47bd-93e4-a886dddf7d6c None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:48.826531 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205848c8-8dbc-47bd-93e4-a886dddf7d6c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:48.827043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205848c8-8dbc-47bd-93e4-a886dddf7d6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:48.828622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205848c8-8dbc-47bd-93e4-a886dddf7d6c None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:48.838480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 455/930] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Jun 27 18:27:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:48.841197 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904c8cd-2c9f-47c8-947d-de0b134c38b5 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:48.930663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:48.947904 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904c8cd-2c9f-47c8-947d-de0b134c38b5 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:48.984372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93473168-cb33-44cf-bbf9-f19204309e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:48.985030 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93473168-cb33-44cf-bbf9-f19204309e80 None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:48.985314 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93473168-cb33-44cf-bbf9-f19204309e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:48.985592 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93473168-cb33-44cf-bbf9-f19204309e80 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:48.986344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93473168-cb33-44cf-bbf9-f19204309e80 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:48.987747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93473168-cb33-44cf-bbf9-f19204309e80 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:49.091016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 476/931] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Jun 27 18:27:48 2022] POST /identity/v3/endpoints => generated 347 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:27:49.108961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5685ae5-65f9-4e74-8496-7994fc99a678 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:49.109962 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5685ae5-65f9-4e74-8496-7994fc99a678 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:49.110532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5685ae5-65f9-4e74-8496-7994fc99a678 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:49.112595 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 456/932] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:49.119198 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2182bb2-5c94-463f-8bb2-f9efd52fb5a9 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:49.119198 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2182bb2-5c94-463f-8bb2-f9efd52fb5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:49.119198 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2182bb2-5c94-463f-8bb2-f9efd52fb5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:49.137480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-a2182bb2-5c94-463f-8bb2-f9efd52fb5a9 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:49.138731 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-a2182bb2-5c94-463f-8bb2-f9efd52fb5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:49.223282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 477/933] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:49.247907 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5685ae5-65f9-4e74-8496-7994fc99a678 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:49.358674 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5685ae5-65f9-4e74-8496-7994fc99a678 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:49.384052 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:49.394702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e981d9f-4d23-4d06-a25d-52bb17f070e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:49.395483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e981d9f-4d23-4d06-a25d-52bb17f070e2 None admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:49.395866 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e981d9f-4d23-4d06-a25d-52bb17f070e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:49.396242 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e981d9f-4d23-4d06-a25d-52bb17f070e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:49.397140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e981d9f-4d23-4d06-a25d-52bb17f070e2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:49.398714 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e981d9f-4d23-4d06-a25d-52bb17f070e2 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:49.400766 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: WARNING keystone.server.flask.common [None req-8e981d9f-4d23-4d06-a25d-52bb17f070e2 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 27 18:27:49.444697 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.sql.core [None req-8e981d9f-4d23-4d06-a25d-52bb17f070e2 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 27 18:27:49.444697 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 27 18:27:49.444697 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [parameters: {'id': 'fa1187f5f6964f38b123bf8493967317', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 27 18:27:49.444697 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80569) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 27 18:27:49.462074 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-8e981d9f-4d23-4d06-a25d-52bb17f070e2 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fa1187f5f6964f38b123bf8493967317.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fa1187f5f6964f38b123bf8493967317. Jun 27 18:27:49.462074 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:49.462074 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 18:27:49.462074 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 18:27:49.462074 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 18:27:49.462074 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 18:27:49.462074 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 18:27:49.462074 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 18:27:49.462074 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 18:27:49.462074 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application conn.query(q) Jun 27 18:27:49.462074 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 18:27:49.462074 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 18:27:49.462074 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 18:27:49.463980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application result.read() Jun 27 18:27:49.463980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 18:27:49.463980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 18:27:49.463980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 18:27:49.463980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 18:27:49.463980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 18:27:49.463980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 18:27:49.463980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 18:27:49.463980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 18:27:49.463980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 27 18:27:49.463980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:49.463980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 27 18:27:49.463980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:49.463980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:49.463980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 27 18:27:49.463980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 27 18:27:49.463980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 27 18:27:49.463980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 27 18:27:49.465555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 18:27:49.465555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application next(self.gen) Jun 27 18:27:49.465555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 27 18:27:49.465555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application yield resource Jun 27 18:27:49.465555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 18:27:49.465555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application next(self.gen) Jun 27 18:27:49.465555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 27 18:27:49.465555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self.session.rollback() Jun 27 18:27:49.465555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 27 18:27:49.465555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self.force_reraise() Jun 27 18:27:49.465555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 27 18:27:49.465555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise self.value Jun 27 18:27:49.465555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 27 18:27:49.465555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 27 18:27:49.465555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 27 18:27:49.465555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application session.commit() Jun 27 18:27:49.465555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 27 18:27:49.466503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 27 18:27:49.466503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 27 18:27:49.466503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._prepare_impl() Jun 27 18:27:49.466503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 27 18:27:49.466503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self.session.flush() Jun 27 18:27:49.466503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 27 18:27:49.466503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._flush(objects) Jun 27 18:27:49.466503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 27 18:27:49.466503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 27 18:27:49.466503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 27 18:27:49.466503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application compat.raise_( Jun 27 18:27:49.466503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 18:27:49.466503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception Jun 27 18:27:49.466503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 27 18:27:49.466503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application flush_context.execute() Jun 27 18:27:49.466503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 27 18:27:49.466503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rec.execute(self) Jun 27 18:27:49.467395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 27 18:27:49.467395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 27 18:27:49.467395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 27 18:27:49.467395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 27 18:27:49.467395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 27 18:27:49.467395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 27 18:27:49.467395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 27 18:27:49.467395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 27 18:27:49.467395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 27 18:27:49.467395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 27 18:27:49.467395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 27 18:27:49.467395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 27 18:27:49.467395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 27 18:27:49.467395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 27 18:27:49.467395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 27 18:27:49.468248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 27 18:27:49.468248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 18:27:49.468248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception Jun 27 18:27:49.468248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 18:27:49.468248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 18:27:49.468248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 18:27:49.468248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 18:27:49.468248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 18:27:49.468248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 18:27:49.468248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 18:27:49.468248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application conn.query(q) Jun 27 18:27:49.468248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 18:27:49.468248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 18:27:49.468248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 18:27:49.468248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application result.read() Jun 27 18:27:49.468248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 18:27:49.468248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 18:27:49.470447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 18:27:49.470447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 18:27:49.470447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 18:27:49.470447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 18:27:49.470447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 18:27:49.470447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 18:27:49.470447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 27 18:27:49.470447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 27 18:27:49.470447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [parameters: {'id': 'fa1187f5f6964f38b123bf8493967317', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 27 18:27:49.470447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 27 18:27:49.470447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:49.470447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:49.470447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:49.470447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:49.470447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 27 18:27:49.472516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 27 18:27:49.472516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 27 18:27:49.472516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 27 18:27:49.472516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 27 18:27:49.472516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:49.472516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:49.472516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:49.472516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:49.472516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:49.472516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:49.472516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:49.472516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:49.472516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:49.472516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:49.472516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:49.472516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:49.472516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:49.474847 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:49.474847 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 27 18:27:49.474847 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 27 18:27:49.474847 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:49.474847 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:49.474847 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 27 18:27:49.474847 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 27 18:27:49.474847 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fa1187f5f6964f38b123bf8493967317. Jun 27 18:27:49.474847 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:49.474847 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 457/934] 10.222.0.15 () {60 vars in 1140 bytes} [Mon Jun 27 18:27:49 2022] POST /identity/v3/projects => generated 262 bytes in 217 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:49.476145 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2182bb2-5c94-463f-8bb2-f9efd52fb5a9 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:49.538332 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:49.574619 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2182bb2-5c94-463f-8bb2-f9efd52fb5a9 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:49.610668 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c752f-edc0-4a41-a1f9-d007f0f49377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:49.611380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c752f-edc0-4a41-a1f9-d007f0f49377 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:49.611663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c752f-edc0-4a41-a1f9-d007f0f49377 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:49.611941 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c752f-edc0-4a41-a1f9-d007f0f49377 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:49.619553 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c752f-edc0-4a41-a1f9-d007f0f49377 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:49.621123 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c752f-edc0-4a41-a1f9-d007f0f49377 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:49.636064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-092c752f-edc0-4a41-a1f9-d007f0f49377 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:49.636064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:49.636064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:49.636064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:49.636064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:49.636064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:49.636064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:49.636064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:49.636064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:49.636064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:49.636064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:49.636064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:49.636064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:49.636064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:49.636064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:49.638249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:49.638249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:49.638249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:49.638249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:49.638249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:49.638249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:49.638249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:49.638249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:49.638249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:49.638249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:49.638249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:49.638249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:49.638249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:49.638249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:49.638249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:49.638249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:49.638249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:49.640415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:49.640415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:49.640415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:49.640415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:49.640415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:49.640415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:49.640415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:49.640415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:49.640415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 478/935] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:49.643397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e981d9f-4d23-4d06-a25d-52bb17f070e2 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:49.756694 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e981d9f-4d23-4d06-a25d-52bb17f070e2 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:49.766585 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d323b68-5b5f-47d2-b0c3-1046ba72e3da None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:49.766875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d323b68-5b5f-47d2-b0c3-1046ba72e3da None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:49.767158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d323b68-5b5f-47d2-b0c3-1046ba72e3da None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:49.768773 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 479/936] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:27:49.773705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e014ab-8a90-4f9b-9774-3af999acd62c None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:49.774004 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e014ab-8a90-4f9b-9774-3af999acd62c None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:49.774288 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e014ab-8a90-4f9b-9774-3af999acd62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:49.792866 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-c2e014ab-8a90-4f9b-9774-3af999acd62c None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:49.794068 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c2e014ab-8a90-4f9b-9774-3af999acd62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:49.794372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbbd69-21cc-4b95-8d10-4103d3d0a60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:49.795074 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbbd69-21cc-4b95-8d10-4103d3d0a60a None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:49.795357 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbbd69-21cc-4b95-8d10-4103d3d0a60a None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:49.795626 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbbd69-21cc-4b95-8d10-4103d3d0a60a None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:49.796683 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfbbd69-21cc-4b95-8d10-4103d3d0a60a None admin] RBAC: Adding query filter params (name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:49.797951 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfbbd69-21cc-4b95-8d10-4103d3d0a60a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:49.799705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfbbd69-21cc-4b95-8d10-4103d3d0a60a None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:49.828113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 458/937] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Jun 27 18:27:49 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:49.875190 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 480/938] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 18:27:49.897741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbbd69-21cc-4b95-8d10-4103d3d0a60a None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:49.943558 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:50.009678 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbbd69-21cc-4b95-8d10-4103d3d0a60a None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:50.045191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f54cde-887c-4474-91a6-ba08489921c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:50.046061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f54cde-887c-4474-91a6-ba08489921c0 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:50.046341 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f54cde-887c-4474-91a6-ba08489921c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:50.046641 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f54cde-887c-4474-91a6-ba08489921c0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:50.058472 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f54cde-887c-4474-91a6-ba08489921c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:50.066549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-e1f54cde-887c-4474-91a6-ba08489921c0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 18:27:50.066549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:50.066549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:50.066549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:50.066549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:50.066549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:50.066549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:50.066549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:50.066549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:50.066549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:50.066549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:50.066549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:50.066549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:50.066549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:50.066549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:50.066549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:50.067483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:50.067483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:50.067483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:50.067483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:50.067483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:50.067483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:50.067483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:50.067483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:50.067483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:50.067483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:50.067483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:50.067483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:50.067483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:50.067483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:50.067483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:50.067483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:50.068331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:50.068331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:50.068331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:50.068331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:50.068331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:50.068331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:50.068331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:50.068331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:50.068331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:50.068331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:50.068331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 18:27:50.068331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:50.068331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 459/939] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Jun 27 18:27:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:50.074410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e014ab-8a90-4f9b-9774-3af999acd62c None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:50.132467 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:50.181329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e014ab-8a90-4f9b-9774-3af999acd62c None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:50.217044 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a64fc3-9414-4978-a82b-1aa0b139c636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:50.217723 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a64fc3-9414-4978-a82b-1aa0b139c636 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:50.218005 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a64fc3-9414-4978-a82b-1aa0b139c636 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:50.218281 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a64fc3-9414-4978-a82b-1aa0b139c636 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:50.219064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a64fc3-9414-4978-a82b-1aa0b139c636 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:50.219374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a64fc3-9414-4978-a82b-1aa0b139c636 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:50.220811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a64fc3-9414-4978-a82b-1aa0b139c636 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:50.230284 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 481/940] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Jun 27 18:27:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:50.235714 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f54cde-887c-4474-91a6-ba08489921c0 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:50.349704 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f54cde-887c-4474-91a6-ba08489921c0 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:50.364096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:50.391123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f37743-536b-4161-bd40-a5fa95b6770c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:50.391923 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f37743-536b-4161-bd40-a5fa95b6770c None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:50.392494 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f37743-536b-4161-bd40-a5fa95b6770c None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:50.392769 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f37743-536b-4161-bd40-a5fa95b6770c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:50.404483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f37743-536b-4161-bd40-a5fa95b6770c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:50.405995 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f37743-536b-4161-bd40-a5fa95b6770c None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:50.424042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-20f37743-536b-4161-bd40-a5fa95b6770c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:50.424042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:50.424042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:50.424042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:50.424042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:50.424042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:50.424042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:50.424042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:50.424042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:50.424042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:50.424042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:50.424042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:50.424042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:50.424042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:50.424042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:50.424961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:50.424961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:50.424961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:50.424961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:50.424961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:50.424961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:50.424961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:50.424961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:50.424961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:50.424961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:50.424961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:50.424961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:50.424961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:50.424961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:50.424961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:50.424961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:50.424961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:50.425863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:50.425863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:50.425863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:50.425863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:50.425863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 460/941] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:50.431937 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a64fc3-9414-4978-a82b-1aa0b139c636 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:50.536877 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a64fc3-9414-4978-a82b-1aa0b139c636 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:50.554159 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:50.571268 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b928685-f8dd-46c9-a2c2-dc422b95654b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:50.572516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b928685-f8dd-46c9-a2c2-dc422b95654b None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:50.572887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b928685-f8dd-46c9-a2c2-dc422b95654b None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:50.573162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b928685-f8dd-46c9-a2c2-dc422b95654b None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:50.573889 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b928685-f8dd-46c9-a2c2-dc422b95654b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:50.574195 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b928685-f8dd-46c9-a2c2-dc422b95654b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:50.575590 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b928685-f8dd-46c9-a2c2-dc422b95654b None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:50.588570 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 482/942] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:50.595760 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f37743-536b-4161-bd40-a5fa95b6770c None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:50.636848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69424f45-939d-4cfd-a01e-3d14526f5850 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:50.637141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69424f45-939d-4cfd-a01e-3d14526f5850 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:50.637409 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69424f45-939d-4cfd-a01e-3d14526f5850 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:50.639049 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 483/943] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:27:50.644465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389910e-745b-40ce-9d71-1c79985eb467 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:50.644995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389910e-745b-40ce-9d71-1c79985eb467 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:50.645501 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389910e-745b-40ce-9d71-1c79985eb467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:50.663116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-b389910e-745b-40ce-9d71-1c79985eb467 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:50.664640 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-b389910e-745b-40ce-9d71-1c79985eb467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:50.702918 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f37743-536b-4161-bd40-a5fa95b6770c None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:50.754400 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 484/944] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 18:27:50.757000 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d6fb6-0074-4460-a826-c7c09fefb7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:50.757853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d6fb6-0074-4460-a826-c7c09fefb7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:50.758136 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d6fb6-0074-4460-a826-c7c09fefb7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:50.758424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d6fb6-0074-4460-a826-c7c09fefb7d7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:50.766452 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d6fb6-0074-4460-a826-c7c09fefb7d7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:50.768003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d6fb6-0074-4460-a826-c7c09fefb7d7 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:50.773191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389910e-745b-40ce-9d71-1c79985eb467 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:50.787135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-947d6fb6-0074-4460-a826-c7c09fefb7d7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 18:27:50.787135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:50.787135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:50.787135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:50.787135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:50.787135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:50.787135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:50.787135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:50.787135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:50.787135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:50.787135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:50.787135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:50.787135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:50.787135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:50.787135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:50.787135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:50.788129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:50.788129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:50.788129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:50.788129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:50.788129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:50.788129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:50.788129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:50.788129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:50.788129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:50.788129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:50.788129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:50.788129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:50.788129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:50.788129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:50.788129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:50.788129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:50.788129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:50.788129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:50.789772 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:50.789772 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:50.789772 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:50.789772 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:50.789772 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:50.789772 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:50.789772 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:50.789772 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:50.789772 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 18:27:50.789772 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:50.789772 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 461/945] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Jun 27 18:27:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:50.795090 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d6fb6-0074-4460-a826-c7c09fefb7d7 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:50.811522 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:50.887190 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389910e-745b-40ce-9d71-1c79985eb467 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:50.898775 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:50.925425 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a09004-8c02-4b15-a1df-a4e0de71b733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:50.926619 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a09004-8c02-4b15-a1df-a4e0de71b733 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:50.927116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a09004-8c02-4b15-a1df-a4e0de71b733 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:50.927816 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a09004-8c02-4b15-a1df-a4e0de71b733 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:50.928717 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a09004-8c02-4b15-a1df-a4e0de71b733 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:50.930457 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a09004-8c02-4b15-a1df-a4e0de71b733 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:50.935774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d6fb6-0074-4460-a826-c7c09fefb7d7 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:50.944541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-a7a09004-8c02-4b15-a1df-a4e0de71b733 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 18:27:50.944541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:50.944541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:27:50.944541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:27:50.944541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:27:50.944541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:27:50.944541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:27:50.944541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:50.944541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 18:27:50.944541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:50.944541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:50.944541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:50.944541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:50.944541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:50.944541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:50.944541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:50.944541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:50.945531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:50.945531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:50.945531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:50.945531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:50.945531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:50.945531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:50.945531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:27:50.945531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:27:50.945531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:27:50.945531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:27:50.945531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:50.945531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:50.945531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:50.945531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:50.945531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:50.945531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:50.946365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:50.946365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:50.946365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:50.946365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:50.946365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:50.946365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:50.946365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:50.946365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:50.946365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:50.946365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:50.946365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:27:50.946365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:50.946365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 18:27:50.946365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:50.954114 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 485/946] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Jun 27 18:27:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:27:50.955610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a09004-8c02-4b15-a1df-a4e0de71b733 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:50.975019 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8c149-3737-4b64-b7c3-8620f17fe5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:50.975684 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8c149-3737-4b64-b7c3-8620f17fe5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:50.975965 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8c149-3737-4b64-b7c3-8620f17fe5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:50.976470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8c149-3737-4b64-b7c3-8620f17fe5a9 None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:50.977322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa8c149-3737-4b64-b7c3-8620f17fe5a9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:50.978046 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa8c149-3737-4b64-b7c3-8620f17fe5a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:50.979515 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa8c149-3737-4b64-b7c3-8620f17fe5a9 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:51.008502 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 462/947] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Jun 27 18:27:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:51.015922 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8c149-3737-4b64-b7c3-8620f17fe5a9 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:51.076555 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a09004-8c02-4b15-a1df-a4e0de71b733 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:51.087228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:51.117577 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f08da2-02cf-430a-8571-1d582352a853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:51.118973 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f08da2-02cf-430a-8571-1d582352a853 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:51.119562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f08da2-02cf-430a-8571-1d582352a853 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:51.120046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f08da2-02cf-430a-8571-1d582352a853 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:51.120978 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f08da2-02cf-430a-8571-1d582352a853 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:51.121506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f08da2-02cf-430a-8571-1d582352a853 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:51.123341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f08da2-02cf-430a-8571-1d582352a853 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:51.125239 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8c149-3737-4b64-b7c3-8620f17fe5a9 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:51.133612 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 486/948] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Jun 27 18:27:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:51.135629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f08da2-02cf-430a-8571-1d582352a853 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:51.162011 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76060b7-4d82-430c-973c-065c171974fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:51.162759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76060b7-4d82-430c-973c-065c171974fe None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:51.163041 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76060b7-4d82-430c-973c-065c171974fe None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:51.163309 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76060b7-4d82-430c-973c-065c171974fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:51.174407 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76060b7-4d82-430c-973c-065c171974fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:51.174839 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:51.176237 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76060b7-4d82-430c-973c-065c171974fe None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:51.194412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-f76060b7-4d82-430c-973c-065c171974fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:51.194412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:51.194412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:51.194412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:51.194412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:51.194412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:51.194412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:51.194412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:51.194412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:51.194412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:51.194412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:51.194412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:51.194412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:51.194412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:51.194412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:51.195331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:51.195331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:51.195331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:51.195331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:51.195331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:51.195331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:51.195331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:51.195331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:51.195331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:51.195331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:51.195331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:51.195331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:51.195331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:51.195331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:51.195331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:51.195331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:51.195331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:51.197211 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:51.197211 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:51.197211 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:51.197211 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:51.201033 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 463/949] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:51.202078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76060b7-4d82-430c-973c-065c171974fe None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:51.250794 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f08da2-02cf-430a-8571-1d582352a853 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:51.287819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2c676-c243-4d05-ab48-1cc5fe1a0ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:51.288894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2c676-c243-4d05-ab48-1cc5fe1a0ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:51.289475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2c676-c243-4d05-ab48-1cc5fe1a0ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:51.289979 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2c676-c243-4d05-ab48-1cc5fe1a0ff1 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:51.291434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2c676-c243-4d05-ab48-1cc5fe1a0ff1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:51.291965 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2c676-c243-4d05-ab48-1cc5fe1a0ff1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:51.293534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2c676-c243-4d05-ab48-1cc5fe1a0ff1 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:51.303961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 487/950] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Jun 27 18:27:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:51.305540 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2c676-c243-4d05-ab48-1cc5fe1a0ff1 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:51.306856 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76060b7-4d82-430c-973c-065c171974fe None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:51.330770 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:51.346528 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ea34c-b4fe-464f-abe7-adcbffe8952a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:51.347956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ea34c-b4fe-464f-abe7-adcbffe8952a None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:51.348238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ea34c-b4fe-464f-abe7-adcbffe8952a None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:51.348513 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ea34c-b4fe-464f-abe7-adcbffe8952a None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:51.349238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ea34c-b4fe-464f-abe7-adcbffe8952a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:51.349542 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ea34c-b4fe-464f-abe7-adcbffe8952a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:51.350929 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ea34c-b4fe-464f-abe7-adcbffe8952a None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:51.368003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 464/951] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:51.369965 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5aa2d-2628-4aef-b734-5b92e7dbec68 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:51.370628 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5aa2d-2628-4aef-b734-5b92e7dbec68 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:51.371310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5aa2d-2628-4aef-b734-5b92e7dbec68 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:51.372785 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 465/952] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:51.377262 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5aa2d-2628-4aef-b734-5b92e7dbec68 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:51.418750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:51.421722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2c676-c243-4d05-ab48-1cc5fe1a0ff1 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:51.457507 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0ecb0-ea4e-4b1c-9b8d-dbeb7b3df19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:51.458249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0ecb0-ea4e-4b1c-9b8d-dbeb7b3df19d None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:51.458624 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0ecb0-ea4e-4b1c-9b8d-dbeb7b3df19d None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:51.459085 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0ecb0-ea4e-4b1c-9b8d-dbeb7b3df19d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:51.459894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0ecb0-ea4e-4b1c-9b8d-dbeb7b3df19d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:51.461459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0ecb0-ea4e-4b1c-9b8d-dbeb7b3df19d None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:51.483334 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5aa2d-2628-4aef-b734-5b92e7dbec68 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:51.516633 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca262730-4206-4718-8130-7b8f204c3e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:51.517438 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca262730-4206-4718-8130-7b8f204c3e4e None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:51.517714 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca262730-4206-4718-8130-7b8f204c3e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:51.517984 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca262730-4206-4718-8130-7b8f204c3e4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:51.524677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca262730-4206-4718-8130-7b8f204c3e4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:51.526227 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca262730-4206-4718-8130-7b8f204c3e4e None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:51.540013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-ca262730-4206-4718-8130-7b8f204c3e4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:51.540013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:51.540013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:51.540013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:51.540013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:51.540013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:51.540013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:51.540013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:51.540013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:51.540013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:51.540013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:51.540013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:51.540013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:51.540013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:51.540013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:51.541276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:51.541276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:51.541276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:51.541276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:51.541276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:51.541276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:51.541276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:51.541276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:51.541276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:51.541276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:51.541276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:51.541276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:51.541276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:51.541276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:51.541276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:51.541276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:51.541276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:51.542288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:51.542288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:51.542288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:51.542288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:51.542288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:51.542288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:51.542288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:51.542288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:51.542288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 466/953] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:51.547387 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f853202-967a-4a13-80e4-80e9eb51e678 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:51.547673 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f853202-967a-4a13-80e4-80e9eb51e678 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:51.547947 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f853202-967a-4a13-80e4-80e9eb51e678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:51.554984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 488/954] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Jun 27 18:27:51 2022] POST /identity/v3/endpoints => generated 349 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:27:51.556554 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0ecb0-ea4e-4b1c-9b8d-dbeb7b3df19d None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:51.567416 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-2f853202-967a-4a13-80e4-80e9eb51e678 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:51.569540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-2f853202-967a-4a13-80e4-80e9eb51e678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:51.623620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:51.670721 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 467/955] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:51.684546 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0ecb0-ea4e-4b1c-9b8d-dbeb7b3df19d None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:51.694630 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f853202-967a-4a13-80e4-80e9eb51e678 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:51.723941 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb835d-5730-4893-b853-eac4111a212c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:51.724885 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb835d-5730-4893-b853-eac4111a212c None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:51.725220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb835d-5730-4893-b853-eac4111a212c None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:51.725556 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb835d-5730-4893-b853-eac4111a212c None admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:51.726558 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb835d-5730-4893-b853-eac4111a212c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:51.726879 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb835d-5730-4893-b853-eac4111a212c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:51.728679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb835d-5730-4893-b853-eac4111a212c None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:51.746853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 489/956] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:51.751713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb835d-5730-4893-b853-eac4111a212c None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:51.774154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:51.843188 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f853202-967a-4a13-80e4-80e9eb51e678 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:51.871057 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb835d-5730-4893-b853-eac4111a212c None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:51.882493 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fd70d-5036-46af-ba54-75971b20205a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:51.883488 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fd70d-5036-46af-ba54-75971b20205a None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:51.883675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fd70d-5036-46af-ba54-75971b20205a None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:51.883933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fd70d-5036-46af-ba54-75971b20205a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:51.885111 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fd70d-5036-46af-ba54-75971b20205a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:51.886492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fd70d-5036-46af-ba54-75971b20205a None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:51.899717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-9f5fd70d-5036-46af-ba54-75971b20205a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 18:27:51.899717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:51.899717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:27:51.899717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:27:51.899717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:27:51.899717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:27:51.899717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:27:51.899717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:51.899717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 18:27:51.899717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:51.899717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:51.899717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:51.899717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:51.899717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:51.899717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:51.899717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:51.899717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:51.901032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:51.901032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:51.901032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:51.901032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:51.901032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:51.901032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:51.901032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:27:51.901032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:27:51.901032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:27:51.901032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:27:51.901032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:51.901032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:51.901032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:51.901032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:51.901032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:51.901032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:51.902397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:51.902397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:51.902397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:51.902397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:51.902397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:51.902397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:51.902397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:51.902397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:51.902397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:51.902397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:51.902397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:27:51.902397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:51.902397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 18:27:51.902397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:51.902397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 468/957] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:27:51.907207 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fd70d-5036-46af-ba54-75971b20205a None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:51.917748 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d06adf-c41a-41b1-b56a-daf92e8dd68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:51.918767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d06adf-c41a-41b1-b56a-daf92e8dd68d None admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:51.919051 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d06adf-c41a-41b1-b56a-daf92e8dd68d None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:51.919323 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d06adf-c41a-41b1-b56a-daf92e8dd68d None admin] PATH_INFO: `/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/58e1f1ac6ea14da0be6e9a24d007847a/roles/3ef93c53540740d6b89875c0ff507fab` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:51.922090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d06adf-c41a-41b1-b56a-daf92e8dd68d None admin] RBAC: Authorizing `identity:create_grant(project_id=84ecf552f8364884b4aeff3428cb81e4, user_id=58e1f1ac6ea14da0be6e9a24d007847a, role_id=3ef93c53540740d6b89875c0ff507fab)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:51.923716 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d06adf-c41a-41b1-b56a-daf92e8dd68d None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:51.945275 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:51.990535 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 490/958] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:27:51 2022] PUT /identity/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/58e1f1ac6ea14da0be6e9a24d007847a/roles/3ef93c53540740d6b89875c0ff507fab => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:27:52.029709 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fd70d-5036-46af-ba54-75971b20205a None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:52.043698 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268394e-9993-42f1-9d47-13816a80efb1 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:52.043980 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268394e-9993-42f1-9d47-13816a80efb1 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:52.044259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268394e-9993-42f1-9d47-13816a80efb1 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:52.045987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 491/959] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:52.051586 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c2118-7866-42df-a2ce-8d47c9ef5573 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:52.051860 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c2118-7866-42df-a2ce-8d47c9ef5573 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:52.052140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c2118-7866-42df-a2ce-8d47c9ef5573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:52.071449 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-1d6c2118-7866-42df-a2ce-8d47c9ef5573 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:52.072901 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6202a-ea00-4735-9e97-b6c4dff27379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:52.073799 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6202a-ea00-4735-9e97-b6c4dff27379 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:52.073927 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-1d6c2118-7866-42df-a2ce-8d47c9ef5573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:52.074406 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6202a-ea00-4735-9e97-b6c4dff27379 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:52.074885 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6202a-ea00-4735-9e97-b6c4dff27379 None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:52.075797 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6202a-ea00-4735-9e97-b6c4dff27379 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:52.076311 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6202a-ea00-4735-9e97-b6c4dff27379 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:52.077971 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6202a-ea00-4735-9e97-b6c4dff27379 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:52.084382 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 469/960] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Jun 27 18:27:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:52.091489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6202a-ea00-4735-9e97-b6c4dff27379 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:52.161207 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 492/961] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 18:27:52.186009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c2118-7866-42df-a2ce-8d47c9ef5573 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:52.195277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:52.220386 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:52.227268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6202a-ea00-4735-9e97-b6c4dff27379 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:52.265217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939dfcd-b9ef-4c4d-8479-d4fe736af194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:52.265986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939dfcd-b9ef-4c4d-8479-d4fe736af194 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:52.266342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939dfcd-b9ef-4c4d-8479-d4fe736af194 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:52.266713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939dfcd-b9ef-4c4d-8479-d4fe736af194 None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:52.267504 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939dfcd-b9ef-4c4d-8479-d4fe736af194 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:52.267889 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939dfcd-b9ef-4c4d-8479-d4fe736af194 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:52.269418 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939dfcd-b9ef-4c4d-8479-d4fe736af194 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:52.275831 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 470/962] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Jun 27 18:27:52 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:52.318547 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c2118-7866-42df-a2ce-8d47c9ef5573 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:52.360492 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb3110-41e7-4230-a8e2-bd26f13645af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:52.361176 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb3110-41e7-4230-a8e2-bd26f13645af None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:52.361457 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb3110-41e7-4230-a8e2-bd26f13645af None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:52.361727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb3110-41e7-4230-a8e2-bd26f13645af None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:52.362612 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb3110-41e7-4230-a8e2-bd26f13645af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:52.364279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb3110-41e7-4230-a8e2-bd26f13645af None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:52.365611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: WARNING keystone.server.flask.common [None req-0afb3110-41e7-4230-a8e2-bd26f13645af None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 27 18:27:52.410507 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.sql.core [None req-0afb3110-41e7-4230-a8e2-bd26f13645af None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 27 18:27:52.410507 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 27 18:27:52.410507 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [parameters: {'user_id': '239eab6647fb4c99a44526534250e271', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 27 18:27:52.410507 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80570) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 27 18:27:52.419344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-0afb3110-41e7-4230-a8e2-bd26f13645af None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 27 18:27:52.419344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:52.419344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 18:27:52.419344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 18:27:52.419344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 18:27:52.419344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 18:27:52.419344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 18:27:52.419344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 18:27:52.419344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 18:27:52.419344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application conn.query(q) Jun 27 18:27:52.419344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 18:27:52.419344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 18:27:52.419344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 18:27:52.419344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application result.read() Jun 27 18:27:52.419344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 18:27:52.420485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 18:27:52.420485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 18:27:52.420485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 18:27:52.420485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 18:27:52.420485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 18:27:52.420485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 18:27:52.420485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 18:27:52.420485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 27 18:27:52.420485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:52.420485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 27 18:27:52.420485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:52.420485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:52.420485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 27 18:27:52.420485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 27 18:27:52.420485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 27 18:27:52.420485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 27 18:27:52.420485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 18:27:52.420485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application next(self.gen) Jun 27 18:27:52.421649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 27 18:27:52.421649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application yield resource Jun 27 18:27:52.421649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 18:27:52.421649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application next(self.gen) Jun 27 18:27:52.421649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 27 18:27:52.421649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self.session.rollback() Jun 27 18:27:52.421649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 27 18:27:52.421649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self.force_reraise() Jun 27 18:27:52.421649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 27 18:27:52.421649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise self.value Jun 27 18:27:52.421649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 27 18:27:52.421649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 27 18:27:52.421649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 27 18:27:52.421649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application session.commit() Jun 27 18:27:52.421649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 27 18:27:52.421649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 27 18:27:52.421649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 27 18:27:52.422772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._prepare_impl() Jun 27 18:27:52.422772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 27 18:27:52.422772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self.session.flush() Jun 27 18:27:52.422772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 27 18:27:52.422772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._flush(objects) Jun 27 18:27:52.422772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 27 18:27:52.422772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 27 18:27:52.422772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 27 18:27:52.422772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application compat.raise_( Jun 27 18:27:52.422772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 18:27:52.422772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception Jun 27 18:27:52.422772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 27 18:27:52.422772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application flush_context.execute() Jun 27 18:27:52.422772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 27 18:27:52.422772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rec.execute(self) Jun 27 18:27:52.422772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 27 18:27:52.422772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 27 18:27:52.423846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 27 18:27:52.423846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 27 18:27:52.423846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 27 18:27:52.423846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 27 18:27:52.423846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 27 18:27:52.423846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 27 18:27:52.423846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 27 18:27:52.423846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 27 18:27:52.423846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 27 18:27:52.423846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 27 18:27:52.423846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 27 18:27:52.423846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 27 18:27:52.423846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 27 18:27:52.423846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 27 18:27:52.423846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 18:27:52.423846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception Jun 27 18:27:52.424887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 18:27:52.424887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 18:27:52.424887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 18:27:52.424887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 18:27:52.424887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 18:27:52.424887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 18:27:52.424887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 18:27:52.424887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application conn.query(q) Jun 27 18:27:52.424887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 18:27:52.424887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 18:27:52.424887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 18:27:52.424887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application result.read() Jun 27 18:27:52.424887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 18:27:52.424887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 18:27:52.424887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 18:27:52.424887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 18:27:52.425896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 18:27:52.425896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 18:27:52.425896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 18:27:52.425896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 18:27:52.425896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 27 18:27:52.425896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 27 18:27:52.425896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [parameters: {'user_id': '239eab6647fb4c99a44526534250e271', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 27 18:27:52.425896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 27 18:27:52.425896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:52.425896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:52.425896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:52.425896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:52.425896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:52.425896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:52.425896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:52.427322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:52.427322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:52.427322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:52.427322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:52.427322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:52.427322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:52.427322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:52.427322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 27 18:27:52.427322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 27 18:27:52.427322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:52.427322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:52.427322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:52.427322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:52.427322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:52.427322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:52.427322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 27 18:27:52.427322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 27 18:27:52.429398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 27 18:27:52.429398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 27 18:27:52.429398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 27 18:27:52.429398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 27 18:27:52.429398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 27 18:27:52.429398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:52.429398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 493/963] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Jun 27 18:27:52 2022] POST /identity/v3/users => generated 160 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:27:52.436388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939dfcd-b9ef-4c4d-8479-d4fe736af194 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:52.550665 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939dfcd-b9ef-4c4d-8479-d4fe736af194 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:52.586498 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9a115-9121-498a-a95b-664459f3a550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:52.588509 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9a115-9121-498a-a95b-664459f3a550 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:52.588796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9a115-9121-498a-a95b-664459f3a550 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:52.589070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9a115-9121-498a-a95b-664459f3a550 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:52.596469 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9a115-9121-498a-a95b-664459f3a550 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:52.597893 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9a115-9121-498a-a95b-664459f3a550 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:52.613002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-fbc9a115-9121-498a-a95b-664459f3a550 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 18:27:52.613002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:52.613002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:52.613002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:52.613002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:52.613002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:52.613002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:52.613002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:52.613002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:52.613002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:52.613002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:52.613002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:52.613002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:52.613002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:52.613002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:52.613002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:52.613985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:52.613985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:52.613985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:52.613985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:52.613985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:52.613985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:52.613985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:52.613985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:52.613985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:52.613985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:52.613985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:52.613985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:52.613985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:52.613985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:52.613985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:52.613985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:52.613985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:52.613985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:52.615094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:52.615094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:52.615094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:52.615094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:52.615094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:52.615094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:52.615094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:52.615094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:52.615094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 18:27:52.615094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:52.618652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 471/964] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:27:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:52.620783 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb3110-41e7-4230-a8e2-bd26f13645af None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:52.693838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:52.729659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb3110-41e7-4230-a8e2-bd26f13645af None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:52.767519 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d246c0-4de1-47af-9ab9-639779cfc5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:52.768294 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d246c0-4de1-47af-9ab9-639779cfc5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:52.768645 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d246c0-4de1-47af-9ab9-639779cfc5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:52.768994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d246c0-4de1-47af-9ab9-639779cfc5a1 None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:52.769885 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d246c0-4de1-47af-9ab9-639779cfc5a1 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:52.770256 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d246c0-4de1-47af-9ab9-639779cfc5a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:52.772010 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d246c0-4de1-47af-9ab9-639779cfc5a1 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:52.812666 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 494/965] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Jun 27 18:27:52 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:53.698239 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff182a0-bee2-4b0c-8dea-43a41aceb2b9 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:53.698794 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff182a0-bee2-4b0c-8dea-43a41aceb2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:53.698794 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff182a0-bee2-4b0c-8dea-43a41aceb2b9 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:53.700318 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 472/966] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:53.705295 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c666c7-bd92-4346-bcb4-6fd379941694 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:53.705672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c666c7-bd92-4346-bcb4-6fd379941694 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:53.705838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c666c7-bd92-4346-bcb4-6fd379941694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:53.723982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-43c666c7-bd92-4346-bcb4-6fd379941694 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:53.725490 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-43c666c7-bd92-4346-bcb4-6fd379941694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:53.802425 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 495/967] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:53.823754 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff182a0-bee2-4b0c-8dea-43a41aceb2b9 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:53.843925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:53.928806 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff182a0-bee2-4b0c-8dea-43a41aceb2b9 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:53.963363 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5a3c24-332d-41b0-af1e-d14e010fe4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:53.964322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a3c24-332d-41b0-af1e-d14e010fe4d7 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:53.964607 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a3c24-332d-41b0-af1e-d14e010fe4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:53.964882 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a3c24-332d-41b0-af1e-d14e010fe4d7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:53.965534 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5a3c24-332d-41b0-af1e-d14e010fe4d7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:53.966925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5a3c24-332d-41b0-af1e-d14e010fe4d7 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:53.979480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-fa5a3c24-332d-41b0-af1e-d14e010fe4d7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 18:27:53.979480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:53.979480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:27:53.979480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:27:53.979480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:27:53.979480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:27:53.979480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:27:53.979480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:53.979480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 18:27:53.979480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:53.979480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:53.979480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:53.979480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:53.979480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:53.979480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:53.979480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:53.979480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:53.980511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:53.980511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:53.980511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:53.980511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:53.980511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:53.980511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:53.980511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:27:53.980511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:27:53.980511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:27:53.980511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:27:53.980511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:53.980511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:53.980511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:53.980511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:53.980511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:53.980511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:53.981635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:53.981635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:53.981635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:53.981635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:53.981635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:53.981635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:53.981635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:53.981635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:53.981635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:53.981635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:53.981635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:27:53.981635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:53.981635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 18:27:53.981635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:53.981635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 473/968] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Jun 27 18:27:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:53.988417 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c666c7-bd92-4346-bcb4-6fd379941694 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:54.047578 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:54.091985 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c666c7-bd92-4346-bcb4-6fd379941694 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:54.111028 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be441f9-cdc9-4656-b425-420504f9dfff None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:54.111317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be441f9-cdc9-4656-b425-420504f9dfff None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:54.111598 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be441f9-cdc9-4656-b425-420504f9dfff None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:54.113218 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 474/969] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:54.117916 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476c27e-dfcb-4482-8be9-a001990594a2 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:54.118206 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476c27e-dfcb-4482-8be9-a001990594a2 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:54.118702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476c27e-dfcb-4482-8be9-a001990594a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:54.129131 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa06cda-16a7-4e36-8c41-dc6fa0e94ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:54.129799 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa06cda-16a7-4e36-8c41-dc6fa0e94ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:54.130570 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa06cda-16a7-4e36-8c41-dc6fa0e94ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:54.130867 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa06cda-16a7-4e36-8c41-dc6fa0e94ae9 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:54.131593 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa06cda-16a7-4e36-8c41-dc6fa0e94ae9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:54.131900 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa06cda-16a7-4e36-8c41-dc6fa0e94ae9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:54.133273 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa06cda-16a7-4e36-8c41-dc6fa0e94ae9 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:54.136157 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-a476c27e-dfcb-4482-8be9-a001990594a2 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:54.137411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-a476c27e-dfcb-4482-8be9-a001990594a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:54.143539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 496/970] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Jun 27 18:27:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:54.144418 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa06cda-16a7-4e36-8c41-dc6fa0e94ae9 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:54.214904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 475/971] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 18:27:54.238948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476c27e-dfcb-4482-8be9-a001990594a2 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:54.254436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:54.257448 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa06cda-16a7-4e36-8c41-dc6fa0e94ae9 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:54.278448 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:54.294702 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3efa3a-401a-49eb-b305-b3025feef949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:54.295363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3efa3a-401a-49eb-b305-b3025feef949 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:54.295646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3efa3a-401a-49eb-b305-b3025feef949 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:54.295920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3efa3a-401a-49eb-b305-b3025feef949 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:54.296626 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3efa3a-401a-49eb-b305-b3025feef949 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:54.296935 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3efa3a-401a-49eb-b305-b3025feef949 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:54.298297 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3efa3a-401a-49eb-b305-b3025feef949 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:54.303297 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 497/972] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Jun 27 18:27:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:54.309483 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3efa3a-401a-49eb-b305-b3025feef949 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:54.345672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476c27e-dfcb-4482-8be9-a001990594a2 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:54.383718 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6a118-0d9f-428d-a5f3-05ed029bf679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:54.384554 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6a118-0d9f-428d-a5f3-05ed029bf679 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:54.384827 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6a118-0d9f-428d-a5f3-05ed029bf679 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:54.385100 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6a118-0d9f-428d-a5f3-05ed029bf679 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:54.395198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b6a118-0d9f-428d-a5f3-05ed029bf679 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:27:54.403326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-28b6a118-0d9f-428d-a5f3-05ed029bf679 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 18:27:54.403326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:54.403326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:54.403326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:54.403326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:54.403326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:54.403326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:54.403326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:54.403326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:54.403326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:54.403326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:54.403326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:54.403326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:27:54.403326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:27:54.403326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:27:54.403326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:27:54.404436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:27:54.404436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:27:54.404436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:27:54.404436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:27:54.404436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:54.404436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:54.404436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:54.404436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:54.404436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:54.404436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:54.404436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:54.404436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:54.404436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:54.404436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:54.404436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:54.404436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:54.405388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:54.405388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:54.405388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:54.405388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:54.405388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:27:54.405388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:27:54.405388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:27:54.405388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:27:54.405388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:27:54.405388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:27:54.405388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 18:27:54.405388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:54.405388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 476/973] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Jun 27 18:27:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:27:54.406183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb933061-3313-4c1f-9a01-cd0b023fa9a8 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:54.406404 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb933061-3313-4c1f-9a01-cd0b023fa9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:54.406691 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb933061-3313-4c1f-9a01-cd0b023fa9a8 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:54.410423 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 477/974] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:54.410423 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb933061-3313-4c1f-9a01-cd0b023fa9a8 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:54.417513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3efa3a-401a-49eb-b305-b3025feef949 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:54.455367 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c4cdd-fe42-4def-bc0c-f2aa183d84a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:54.456017 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c4cdd-fe42-4def-bc0c-f2aa183d84a2 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:54.456300 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c4cdd-fe42-4def-bc0c-f2aa183d84a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:54.456575 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c4cdd-fe42-4def-bc0c-f2aa183d84a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:54.457374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c4cdd-fe42-4def-bc0c-f2aa183d84a2 None admin] RBAC: Adding query filter params (interface=admin, service_id=b48ac84e610b4bd2adf2be27253358d6, region_id=RegionOne) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:54.457684 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c4cdd-fe42-4def-bc0c-f2aa183d84a2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:54.459081 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c4cdd-fe42-4def-bc0c-f2aa183d84a2 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:54.464980 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 498/975] 10.222.0.15 () {56 vars in 1249 bytes} [Mon Jun 27 18:27:54 2022] GET /identity/v3/endpoints?service_id=b48ac84e610b4bd2adf2be27253358d6&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:54.466078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7de5b-c040-4922-82ed-67441b16abb6 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:54.466362 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7de5b-c040-4922-82ed-67441b16abb6 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:54.466673 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7de5b-c040-4922-82ed-67441b16abb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:54.483252 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-d6f7de5b-c040-4922-82ed-67441b16abb6 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:54.484437 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-d6f7de5b-c040-4922-82ed-67441b16abb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:54.487548 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:54.517702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb933061-3313-4c1f-9a01-cd0b023fa9a8 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:54.527981 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:54.550839 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 499/976] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:54.551705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507adab2-eb58-4362-ba6b-d1baa069cd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:54.551833 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7de5b-c040-4922-82ed-67441b16abb6 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:54.552421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507adab2-eb58-4362-ba6b-d1baa069cd5b None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:54.552686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507adab2-eb58-4362-ba6b-d1baa069cd5b None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:54.552962 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507adab2-eb58-4362-ba6b-d1baa069cd5b None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:54.553704 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507adab2-eb58-4362-ba6b-d1baa069cd5b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:54.554013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507adab2-eb58-4362-ba6b-d1baa069cd5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:54.555427 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507adab2-eb58-4362-ba6b-d1baa069cd5b None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:54.566116 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 478/977] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Jun 27 18:27:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:54.572672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507adab2-eb58-4362-ba6b-d1baa069cd5b None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:54.656221 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7de5b-c040-4922-82ed-67441b16abb6 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:54.671523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507adab2-eb58-4362-ba6b-d1baa069cd5b None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:54.689938 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd081e5-bdd1-41c4-8f5f-f6338eb55811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:54.690644 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd081e5-bdd1-41c4-8f5f-f6338eb55811 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:54.690936 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd081e5-bdd1-41c4-8f5f-f6338eb55811 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:54.691214 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd081e5-bdd1-41c4-8f5f-f6338eb55811 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:54.691892 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd081e5-bdd1-41c4-8f5f-f6338eb55811 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:54.693262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd081e5-bdd1-41c4-8f5f-f6338eb55811 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:54.699454 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 500/978] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Jun 27 18:27:54 2022] GET /identity/v3/services => generated 1497 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:27:54.700222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd081e5-bdd1-41c4-8f5f-f6338eb55811 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:54.705394 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe227a5-abce-45d4-9b3a-9dd54304f8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:54.706203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe227a5-abce-45d4-9b3a-9dd54304f8f6 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:54.706521 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe227a5-abce-45d4-9b3a-9dd54304f8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:54.706800 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe227a5-abce-45d4-9b3a-9dd54304f8f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:54.709061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:54.719762 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe227a5-abce-45d4-9b3a-9dd54304f8f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:54.721155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe227a5-abce-45d4-9b3a-9dd54304f8f6 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:54.743956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-afe227a5-abce-45d4-9b3a-9dd54304f8f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:54.743956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:54.743956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:54.743956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:54.743956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:54.743956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:54.743956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:54.743956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:54.743956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:54.743956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:54.743956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:54.743956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:54.743956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:54.743956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:54.743956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:54.745109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:54.745109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:54.745109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:54.745109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:54.745109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:54.745109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:54.745109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:54.745109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:54.745109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:54.745109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:54.745109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:54.745109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:54.745109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:54.745109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:54.745109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:54.745109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:54.745109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:54.746231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:54.746231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:54.746231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:54.746231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:54.747998 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 479/979] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:27:54.755876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe227a5-abce-45d4-9b3a-9dd54304f8f6 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:54.767543 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:54.834925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd081e5-bdd1-41c4-8f5f-f6338eb55811 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:54.874210 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe227a5-abce-45d4-9b3a-9dd54304f8f6 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:54.878590 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd65f5-30a5-4edf-9f4a-c1b411e52428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:54.879257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd65f5-30a5-4edf-9f4a-c1b411e52428 None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:54.879538 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd65f5-30a5-4edf-9f4a-c1b411e52428 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:54.879822 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd65f5-30a5-4edf-9f4a-c1b411e52428 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:54.880548 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd65f5-30a5-4edf-9f4a-c1b411e52428 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:54.881920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd65f5-30a5-4edf-9f4a-c1b411e52428 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:54.910803 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb42582-2ea8-4856-8ecc-2ae2b7750dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:54.911457 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb42582-2ea8-4856-8ecc-2ae2b7750dad None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:54.911741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb42582-2ea8-4856-8ecc-2ae2b7750dad None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:54.912014 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb42582-2ea8-4856-8ecc-2ae2b7750dad None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:54.912740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb42582-2ea8-4856-8ecc-2ae2b7750dad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:54.913038 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb42582-2ea8-4856-8ecc-2ae2b7750dad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:54.914403 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb42582-2ea8-4856-8ecc-2ae2b7750dad None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:54.926271 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 480/980] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:54.931775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb42582-2ea8-4856-8ecc-2ae2b7750dad None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:54.944685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 501/981] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Jun 27 18:27:54 2022] POST /identity/v3/services => generated 242 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:27:54.986931 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:55.046578 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb42582-2ea8-4856-8ecc-2ae2b7750dad None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:55.090348 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077cf4e-4a60-4cf8-af8d-50874bd26a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:55.091268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077cf4e-4a60-4cf8-af8d-50874bd26a9e None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:55.091619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077cf4e-4a60-4cf8-af8d-50874bd26a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:55.091954 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077cf4e-4a60-4cf8-af8d-50874bd26a9e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:55.101633 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077cf4e-4a60-4cf8-af8d-50874bd26a9e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:55.102405 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077cf4e-4a60-4cf8-af8d-50874bd26a9e None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:55.118463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-2077cf4e-4a60-4cf8-af8d-50874bd26a9e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 18:27:55.118463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:55.118463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:55.118463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:55.118463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:55.118463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:55.118463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:55.118463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:55.118463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:55.118463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:55.118463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:55.118463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:55.118463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:27:55.118463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:27:55.118463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:27:55.118463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:27:55.119801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:55.119801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:55.119801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:27:55.119801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:55.119801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:27:55.119801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:27:55.119801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:55.119801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:55.119801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:55.119801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:55.119801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:55.119801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:55.119801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:55.119801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:55.119801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:55.119801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:55.119801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:55.119801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:55.120994 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:55.120994 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:55.120994 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:27:55.120994 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:27:55.120994 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:27:55.120994 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:27:55.120994 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:27:55.120994 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:27:55.120994 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 18:27:55.120994 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:55.121656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 481/982] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Jun 27 18:27:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:55.127289 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd65f5-30a5-4edf-9f4a-c1b411e52428 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:55.234284 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd65f5-30a5-4edf-9f4a-c1b411e52428 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:55.236336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:55.269511 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a378660-9a57-415c-9e7b-3256287ee2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:55.270098 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a378660-9a57-415c-9e7b-3256287ee2cf None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:55.270410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a378660-9a57-415c-9e7b-3256287ee2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:55.270677 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a378660-9a57-415c-9e7b-3256287ee2cf None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:55.271517 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a378660-9a57-415c-9e7b-3256287ee2cf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:55.271825 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a378660-9a57-415c-9e7b-3256287ee2cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:55.273253 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a378660-9a57-415c-9e7b-3256287ee2cf None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:55.292749 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 502/983] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Jun 27 18:27:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:55.297256 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077cf4e-4a60-4cf8-af8d-50874bd26a9e None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:55.398947 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077cf4e-4a60-4cf8-af8d-50874bd26a9e None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:55.401144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:55.437016 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e403119-a548-4139-9cde-7aaa7b1b94ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:55.437755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e403119-a548-4139-9cde-7aaa7b1b94ef None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:55.438109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e403119-a548-4139-9cde-7aaa7b1b94ef None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:55.438483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e403119-a548-4139-9cde-7aaa7b1b94ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:55.506108 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e403119-a548-4139-9cde-7aaa7b1b94ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:55.507691 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e403119-a548-4139-9cde-7aaa7b1b94ef None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:55.609707 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 482/984] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:27:55 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 18:27:55.612552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a378660-9a57-415c-9e7b-3256287ee2cf None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:55.712382 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a378660-9a57-415c-9e7b-3256287ee2cf None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:55.754506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e9428-bfbd-4ff0-b09a-7f18f33b2540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:55.755536 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e9428-bfbd-4ff0-b09a-7f18f33b2540 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:55.756024 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e9428-bfbd-4ff0-b09a-7f18f33b2540 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:55.756503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e9428-bfbd-4ff0-b09a-7f18f33b2540 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:55.767514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e9428-bfbd-4ff0-b09a-7f18f33b2540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:55.769229 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e9428-bfbd-4ff0-b09a-7f18f33b2540 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:55.787473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-b84e9428-bfbd-4ff0-b09a-7f18f33b2540 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:55.787473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:55.787473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:55.787473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:55.787473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:55.787473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:55.787473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:55.787473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:55.787473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:55.787473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:55.787473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:55.787473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:55.787473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:27:55.787473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:27:55.787473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:27:55.788373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:27:55.788373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:55.788373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:55.788373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:55.788373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:55.788373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:55.788373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:55.788373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:55.788373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:55.788373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:55.788373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:55.788373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:55.788373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:55.788373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:55.788373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:55.788373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:55.788373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:55.789258 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:27:55.789258 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:27:55.789258 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:27:55.789258 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:55.791230 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 503/985] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:55.796832 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e403119-a548-4139-9cde-7aaa7b1b94ef None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:55.799517 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:55.900058 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e403119-a548-4139-9cde-7aaa7b1b94ef None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:55.934805 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89ad3a-db35-4a43-8091-dcee73f6870c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:55.935798 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89ad3a-db35-4a43-8091-dcee73f6870c None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:55.936297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89ad3a-db35-4a43-8091-dcee73f6870c None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:55.936782 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89ad3a-db35-4a43-8091-dcee73f6870c None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:55.937708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89ad3a-db35-4a43-8091-dcee73f6870c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:55.938214 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89ad3a-db35-4a43-8091-dcee73f6870c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:55.939783 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89ad3a-db35-4a43-8091-dcee73f6870c None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:55.951729 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 483/986] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:55.957310 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e9428-bfbd-4ff0-b09a-7f18f33b2540 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:55.972766 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:56.057733 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e9428-bfbd-4ff0-b09a-7f18f33b2540 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:56.093236 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49acc6-036e-42a7-ab1b-0c71ecddc6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:56.094024 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49acc6-036e-42a7-ab1b-0c71ecddc6cb None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:56.094301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49acc6-036e-42a7-ab1b-0c71ecddc6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:56.094592 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49acc6-036e-42a7-ab1b-0c71ecddc6cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:56.101225 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49acc6-036e-42a7-ab1b-0c71ecddc6cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:56.102735 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49acc6-036e-42a7-ab1b-0c71ecddc6cb None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:56.116353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-4e49acc6-036e-42a7-ab1b-0c71ecddc6cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:56.116353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:56.116353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:56.116353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:56.116353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:56.116353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:56.116353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:56.116353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:56.116353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:56.116353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:56.116353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:56.116353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:56.116353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:27:56.116353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:27:56.116353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:27:56.117243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:27:56.117243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:56.117243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:56.117243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:56.117243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:56.117243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:56.117243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:56.117243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:56.117243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:56.117243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:56.117243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:56.117243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:56.117243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:56.117243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:56.117243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:56.117243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:56.117243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:56.119215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:27:56.119215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:27:56.119215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:27:56.119215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:27:56.119215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:27:56.119215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:27:56.119215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:27:56.119215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:56.122714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 504/987] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:56.123914 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89ad3a-db35-4a43-8091-dcee73f6870c None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:56.187225 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:56.224960 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89ad3a-db35-4a43-8091-dcee73f6870c None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:56.260177 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf3e2a-7d84-4317-8994-6bf955970971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:56.260832 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf3e2a-7d84-4317-8994-6bf955970971 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:56.261115 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf3e2a-7d84-4317-8994-6bf955970971 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:56.261389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf3e2a-7d84-4317-8994-6bf955970971 None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:56.262169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf3e2a-7d84-4317-8994-6bf955970971 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:56.262505 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf3e2a-7d84-4317-8994-6bf955970971 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:56.264019 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf3e2a-7d84-4317-8994-6bf955970971 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:56.276733 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 484/988] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:27:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:56.282442 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49acc6-036e-42a7-ab1b-0c71ecddc6cb None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:56.355898 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:56.387435 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49acc6-036e-42a7-ab1b-0c71ecddc6cb None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:56.422707 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fcc3d7-ba35-4806-9b5e-f1025bed3fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:56.423354 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcc3d7-ba35-4806-9b5e-f1025bed3fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:56.423632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcc3d7-ba35-4806-9b5e-f1025bed3fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:56.423903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcc3d7-ba35-4806-9b5e-f1025bed3fe6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:56.424728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fcc3d7-ba35-4806-9b5e-f1025bed3fe6 None admin] RBAC: Adding query filter params (role.id=3ef93c53540740d6b89875c0ff507fab, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=58e1f1ac6ea14da0be6e9a24d007847a) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:56.425035 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fcc3d7-ba35-4806-9b5e-f1025bed3fe6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:56.426486 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fcc3d7-ba35-4806-9b5e-f1025bed3fe6 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:56.438689 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 505/989] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:27:56 2022] GET /identity/v3/role_assignments?user.id=58e1f1ac6ea14da0be6e9a24d007847a&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=3ef93c53540740d6b89875c0ff507fab => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:56.771395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0b0a6-e3ac-43ba-a47a-621adfb4d731 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:56.771705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0b0a6-e3ac-43ba-a47a-621adfb4d731 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:56.771969 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0b0a6-e3ac-43ba-a47a-621adfb4d731 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:56.773468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 485/990] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:56.779186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc98e5-ef4a-46a1-b122-2e036fd70f9e None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:56.779634 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc98e5-ef4a-46a1-b122-2e036fd70f9e None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:56.779804 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc98e5-ef4a-46a1-b122-2e036fd70f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:56.797082 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-4ffc98e5-ef4a-46a1-b122-2e036fd70f9e None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:56.798272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-4ffc98e5-ef4a-46a1-b122-2e036fd70f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:56.865685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 506/991] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:56 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:56.888577 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0b0a6-e3ac-43ba-a47a-621adfb4d731 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:56.994768 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0b0a6-e3ac-43ba-a47a-621adfb4d731 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:57.017472 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:57.027132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ba236-ebbc-4008-ac9a-6522d7f89685 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:57.027427 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ba236-ebbc-4008-ac9a-6522d7f89685 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:57.027769 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ba236-ebbc-4008-ac9a-6522d7f89685 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:57.029258 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 507/992] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:57.030834 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf8196-d1f1-44be-9a8a-0b4634c4923d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:57.031636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf8196-d1f1-44be-9a8a-0b4634c4923d None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:57.031917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf8196-d1f1-44be-9a8a-0b4634c4923d None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:57.032196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf8196-d1f1-44be-9a8a-0b4634c4923d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:57.032945 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf8196-d1f1-44be-9a8a-0b4634c4923d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:57.034055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75464e-5bb7-4f61-848d-c3ee4c73b2cc None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:57.034263 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75464e-5bb7-4f61-848d-c3ee4c73b2cc None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:57.034356 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf8196-d1f1-44be-9a8a-0b4634c4923d None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:57.034575 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75464e-5bb7-4f61-848d-c3ee4c73b2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:57.047123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-43cf8196-d1f1-44be-9a8a-0b4634c4923d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 18:27:57.047123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:57.047123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:27:57.047123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:27:57.047123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:27:57.047123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:27:57.047123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:27:57.047123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:57.047123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 18:27:57.047123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:57.047123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:57.047123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:57.047123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:57.047123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:57.047123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:57.047123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:57.047123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:57.048134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:57.048134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:57.048134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:57.048134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:57.048134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:57.048134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:57.048134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:27:57.048134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:27:57.048134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:27:57.048134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:27:57.048134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:57.048134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:57.048134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:57.048134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:57.048134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:57.048134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:57.049171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:57.049171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:57.049171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:57.049171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:57.049171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:57.049171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:57.049171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:57.049171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:57.049171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:57.049171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:57.049171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:27:57.049171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:57.049171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 18:27:57.049171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:57.049171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 486/993] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Jun 27 18:27:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:57.054129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf8196-d1f1-44be-9a8a-0b4634c4923d None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:57.054913 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-7b75464e-5bb7-4f61-848d-c3ee4c73b2cc None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:57.056337 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-7b75464e-5bb7-4f61-848d-c3ee4c73b2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:57.125031 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:57.133584 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 508/994] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 18:27:57.157556 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75464e-5bb7-4f61-848d-c3ee4c73b2cc None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:57.163921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf8196-d1f1-44be-9a8a-0b4634c4923d None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:57.202536 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9d89d-1815-4c22-9181-b5ab900c58ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:57.203500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9d89d-1815-4c22-9181-b5ab900c58ee None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:57.203977 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9d89d-1815-4c22-9181-b5ab900c58ee None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:57.204443 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9d89d-1815-4c22-9181-b5ab900c58ee None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:57.205330 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a9d89d-1815-4c22-9181-b5ab900c58ee None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:57.205818 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a9d89d-1815-4c22-9181-b5ab900c58ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:57.207383 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a9d89d-1815-4c22-9181-b5ab900c58ee None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:57.220044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 487/995] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Jun 27 18:27:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:57.220900 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9d89d-1815-4c22-9181-b5ab900c58ee None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:57.270523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75464e-5bb7-4f61-848d-c3ee4c73b2cc None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:57.274011 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:57.307355 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b1173a-8061-4957-8c15-79ea32160a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:57.308146 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1173a-8061-4957-8c15-79ea32160a55 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:57.308433 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1173a-8061-4957-8c15-79ea32160a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:57.308761 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1173a-8061-4957-8c15-79ea32160a55 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:57.309419 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b1173a-8061-4957-8c15-79ea32160a55 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:57.310827 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b1173a-8061-4957-8c15-79ea32160a55 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:57.323401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-b6b1173a-8061-4957-8c15-79ea32160a55 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 18:27:57.323401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:57.323401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:27:57.323401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:27:57.323401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:27:57.323401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:27:57.323401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:27:57.323401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:57.323401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 18:27:57.323401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:57.323401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:57.323401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:57.323401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:57.323401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:57.323401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:57.323401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:57.323401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:57.329162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:57.329162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:57.329162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:57.329162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:57.329162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:57.329162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:57.329162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:27:57.329162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:27:57.329162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:27:57.329162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:27:57.329162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:57.329162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:57.329162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:57.329162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:57.329162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:57.329162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:57.330220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:57.330220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:57.330220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:57.330220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:57.330220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:57.330220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:57.330220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:57.330220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:57.330220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:57.330220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:57.330220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:27:57.330220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:57.330220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 18:27:57.330220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:27:57.330220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 509/996] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:27:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:27:57.331149 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9d89d-1815-4c22-9181-b5ab900c58ee None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:57.331227 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b1173a-8061-4957-8c15-79ea32160a55 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:57.367241 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e11e4-8bd6-4d81-9503-ecf0de2b1d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:57.367908 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e11e4-8bd6-4d81-9503-ecf0de2b1d28 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:57.368246 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e11e4-8bd6-4d81-9503-ecf0de2b1d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:57.368582 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e11e4-8bd6-4d81-9503-ecf0de2b1d28 None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:57.369406 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3e11e4-8bd6-4d81-9503-ecf0de2b1d28 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:57.369720 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3e11e4-8bd6-4d81-9503-ecf0de2b1d28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:57.371115 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3e11e4-8bd6-4d81-9503-ecf0de2b1d28 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:57.376253 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 488/997] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Jun 27 18:27:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:57.381594 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e11e4-8bd6-4d81-9503-ecf0de2b1d28 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:57.398301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:57.443950 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b1173a-8061-4957-8c15-79ea32160a55 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:57.480663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959310b-22f0-4297-a5ec-2b15ad955a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:57.481572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959310b-22f0-4297-a5ec-2b15ad955a1d None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:57.481870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959310b-22f0-4297-a5ec-2b15ad955a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:57.482134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959310b-22f0-4297-a5ec-2b15ad955a1d None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:57.482866 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959310b-22f0-4297-a5ec-2b15ad955a1d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:57.483175 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959310b-22f0-4297-a5ec-2b15ad955a1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:57.483700 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e11e4-8bd6-4d81-9503-ecf0de2b1d28 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:57.484539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959310b-22f0-4297-a5ec-2b15ad955a1d None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:57.489824 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 510/998] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Jun 27 18:27:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:57.495262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959310b-22f0-4297-a5ec-2b15ad955a1d None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:57.498375 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:57.521548 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136e6cc-eeac-41a6-a14e-2d05a926d007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:57.522120 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136e6cc-eeac-41a6-a14e-2d05a926d007 None admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:57.522425 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136e6cc-eeac-41a6-a14e-2d05a926d007 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:57.522704 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136e6cc-eeac-41a6-a14e-2d05a926d007 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:57.523518 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6136e6cc-eeac-41a6-a14e-2d05a926d007 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:57.525828 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6136e6cc-eeac-41a6-a14e-2d05a926d007 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:57.601292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 489/999] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Jun 27 18:27:57 2022] POST /identity/v3/endpoints => generated 325 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:27:57.611827 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959310b-22f0-4297-a5ec-2b15ad955a1d None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:57.637346 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:57.655739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ca8c4f-398d-4348-be95-068fd24d54fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:57.656414 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca8c4f-398d-4348-be95-068fd24d54fb None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:57.656699 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca8c4f-398d-4348-be95-068fd24d54fb None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:57.656974 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ca8c4f-398d-4348-be95-068fd24d54fb None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:57.657702 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ca8c4f-398d-4348-be95-068fd24d54fb None admin] RBAC: Adding query filter params (type=compute) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:57.658006 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ca8c4f-398d-4348-be95-068fd24d54fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:57.659479 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ca8c4f-398d-4348-be95-068fd24d54fb None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:57.665973 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 511/1000] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Jun 27 18:27:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:57.671711 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136e6cc-eeac-41a6-a14e-2d05a926d007 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:57.791238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136e6cc-eeac-41a6-a14e-2d05a926d007 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:57.825422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4538b8b-5477-44de-a464-7ad5f973f489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:57.826082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4538b8b-5477-44de-a464-7ad5f973f489 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:57.826363 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4538b8b-5477-44de-a464-7ad5f973f489 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:57.826660 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4538b8b-5477-44de-a464-7ad5f973f489 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:57.827441 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4538b8b-5477-44de-a464-7ad5f973f489 None admin] RBAC: Adding query filter params (interface=public, service_id=80a452878af449beb06c890c4c7165ff, region_id=RegionOne) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:57.827748 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4538b8b-5477-44de-a464-7ad5f973f489 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:57.829090 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4538b8b-5477-44de-a464-7ad5f973f489 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:57.835432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 490/1001] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Jun 27 18:27:57 2022] GET /identity/v3/endpoints?service_id=80a452878af449beb06c890c4c7165ff&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:57.840687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ca8c4f-398d-4348-be95-068fd24d54fb None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:57.943463 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ca8c4f-398d-4348-be95-068fd24d54fb None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:57.978279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4919cc-93ca-4565-b475-d2f8127bca99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:57.978973 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4919cc-93ca-4565-b475-d2f8127bca99 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:57.979259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4919cc-93ca-4565-b475-d2f8127bca99 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:57.979537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4919cc-93ca-4565-b475-d2f8127bca99 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:57.980226 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4919cc-93ca-4565-b475-d2f8127bca99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:57.981852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4919cc-93ca-4565-b475-d2f8127bca99 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:57.994465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 512/1002] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Jun 27 18:27:57 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:27:58.496243 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc979f8-2922-4d6b-9863-99915e01a0de None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:58.496766 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc979f8-2922-4d6b-9863-99915e01a0de None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:58.496844 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc979f8-2922-4d6b-9863-99915e01a0de None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:58.498328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 491/1003] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:58.503238 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd671a-29b3-4643-b544-2beea260b146 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:58.503531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd671a-29b3-4643-b544-2beea260b146 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:58.503810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd671a-29b3-4643-b544-2beea260b146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:58.520856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-40dd671a-29b3-4643-b544-2beea260b146 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:58.522035 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-40dd671a-29b3-4643-b544-2beea260b146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:58.566426 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:58.589177 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 513/1004] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:58 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:58.611663 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc979f8-2922-4d6b-9863-99915e01a0de None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:58.654182 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:58.716047 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc979f8-2922-4d6b-9863-99915e01a0de None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:58.761514 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7db7cc-2d41-4410-a525-61ef51c77250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:58.762320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7db7cc-2d41-4410-a525-61ef51c77250 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:58.762633 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7db7cc-2d41-4410-a525-61ef51c77250 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:58.762911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7db7cc-2d41-4410-a525-61ef51c77250 None admin] PATH_INFO: `/v3/services/image` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:58.763576 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7db7cc-2d41-4410-a525-61ef51c77250 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:58.764963 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7db7cc-2d41-4410-a525-61ef51c77250 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:58.777411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-cc7db7cc-2d41-4410-a525-61ef51c77250 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 18:27:58.777411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:58.777411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:27:58.777411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:27:58.777411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:27:58.777411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:27:58.777411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:27:58.777411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:58.777411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 18:27:58.777411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:58.777411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:27:58.777411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:58.777411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:27:58.777411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:27:58.777411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:27:58.777411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:27:58.777411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:27:58.778543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:27:58.778543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:27:58.778543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:27:58.778543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:27:58.778543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:27:58.778543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:27:58.778543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:27:58.778543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:27:58.778543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:27:58.778543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:27:58.778543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:27:58.778543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:27:58.778543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:27:58.778543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:27:58.778543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:27:58.778543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:27:58.779432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:27:58.779432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:27:58.779432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:27:58.779432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:27:58.779432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:27:58.779432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:27:58.779432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:27:58.779432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:27:58.779432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:27:58.779432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:27:58.779432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:27:58.779432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:27:58.779432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 18:27:58.779432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:27:58.779432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 492/1005] 10.222.0.15 () {56 vars in 1105 bytes} [Mon Jun 27 18:27:58 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:27:58.786708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd671a-29b3-4643-b544-2beea260b146 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:58.887958 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd671a-29b3-4643-b544-2beea260b146 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:58.922098 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3e3dbe-667f-492a-b09d-8f68f47d1e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:58.922778 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e3dbe-667f-492a-b09d-8f68f47d1e6b None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:58.923060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e3dbe-667f-492a-b09d-8f68f47d1e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:58.923333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3e3dbe-667f-492a-b09d-8f68f47d1e6b None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:58.924031 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3e3dbe-667f-492a-b09d-8f68f47d1e6b None admin] RBAC: Adding query filter params (name=image) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:58.924333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3e3dbe-667f-492a-b09d-8f68f47d1e6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:58.925687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3e3dbe-667f-492a-b09d-8f68f47d1e6b None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:58.930701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 514/1006] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:58.936204 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7db7cc-2d41-4410-a525-61ef51c77250 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:59.037986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7db7cc-2d41-4410-a525-61ef51c77250 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:59.039481 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:59.072107 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd673982-eb56-4c8e-9784-25253d705fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:59.072770 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd673982-eb56-4c8e-9784-25253d705fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:59.073056 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd673982-eb56-4c8e-9784-25253d705fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:59.073334 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd673982-eb56-4c8e-9784-25253d705fd9 None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:59.074052 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd673982-eb56-4c8e-9784-25253d705fd9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:27:59.074361 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd673982-eb56-4c8e-9784-25253d705fd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:59.075833 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd673982-eb56-4c8e-9784-25253d705fd9 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:59.086685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 493/1007] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:58 2022] GET /identity/v3/services?type=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:27:59.722808 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405df767-0d1f-4d0f-a7f7-10f773eafb96 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:59.723135 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405df767-0d1f-4d0f-a7f7-10f773eafb96 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:59.723373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405df767-0d1f-4d0f-a7f7-10f773eafb96 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:59.724780 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 515/1008] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:27:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:27:59.729985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804f24a-ce00-455c-9031-01fcf9090362 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:59.730444 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804f24a-ce00-455c-9031-01fcf9090362 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:59.730622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804f24a-ce00-455c-9031-01fcf9090362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:59.748396 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-b804f24a-ce00-455c-9031-01fcf9090362 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:27:59.749477 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-b804f24a-ce00-455c-9031-01fcf9090362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:27:59.817570 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 494/1009] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:27:59 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:27:59.841455 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405df767-0d1f-4d0f-a7f7-10f773eafb96 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:27:59.887632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:27:59.943957 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405df767-0d1f-4d0f-a7f7-10f773eafb96 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:27:59.978006 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35796d3e-78fb-4185-877d-53e7aca2144c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:27:59.978737 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35796d3e-78fb-4185-877d-53e7aca2144c None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:27:59.979015 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35796d3e-78fb-4185-877d-53e7aca2144c None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:27:59.979284 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35796d3e-78fb-4185-877d-53e7aca2144c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:27:59.980453 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35796d3e-78fb-4185-877d-53e7aca2144c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:27:59.982133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35796d3e-78fb-4185-877d-53e7aca2144c None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:27:59.983990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 516/1010] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:27:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:27:59.989408 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804f24a-ce00-455c-9031-01fcf9090362 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:00.004835 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05134858-2840-4628-a2df-c7c4d34af33e None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:00.005114 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05134858-2840-4628-a2df-c7c4d34af33e None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:00.005384 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05134858-2840-4628-a2df-c7c4d34af33e None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:00.006912 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 517/1011] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:28:00.011406 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488960bf-96aa-4b5c-a5c2-c461785b1f2a None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:00.011688 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488960bf-96aa-4b5c-a5c2-c461785b1f2a None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:00.011963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488960bf-96aa-4b5c-a5c2-c461785b1f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:00.028756 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-488960bf-96aa-4b5c-a5c2-c461785b1f2a None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:00.029933 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-488960bf-96aa-4b5c-a5c2-c461785b1f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:00.087776 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:00.094705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804f24a-ce00-455c-9031-01fcf9090362 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:00.101512 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 518/1012] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:00 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 18:28:00.124684 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488960bf-96aa-4b5c-a5c2-c461785b1f2a None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:00.129977 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f02513-783f-450c-acd8-8e8f69c421c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:00.130661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f02513-783f-450c-acd8-8e8f69c421c3 None admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:00.130947 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f02513-783f-450c-acd8-8e8f69c421c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:00.131226 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f02513-783f-450c-acd8-8e8f69c421c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:00.131942 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f02513-783f-450c-acd8-8e8f69c421c3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:00.133368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f02513-783f-450c-acd8-8e8f69c421c3 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:00.167386 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:00.230500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 495/1013] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Jun 27 18:27:59 2022] POST /identity/v3/projects => generated 311 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:28:00.234636 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488960bf-96aa-4b5c-a5c2-c461785b1f2a None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:00.278027 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f1652-075b-4b45-a2df-4153351bcd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:00.278884 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f1652-075b-4b45-a2df-4153351bcd45 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:00.279161 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f1652-075b-4b45-a2df-4153351bcd45 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:00.279442 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f1652-075b-4b45-a2df-4153351bcd45 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:00.280114 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f1652-075b-4b45-a2df-4153351bcd45 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:00.281558 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f1652-075b-4b45-a2df-4153351bcd45 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:00.294649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-528f1652-075b-4b45-a2df-4153351bcd45 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 18:28:00.294649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:00.294649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:28:00.294649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:28:00.294649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:28:00.294649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:28:00.294649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:28:00.294649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:28:00.294649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 18:28:00.294649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:28:00.294649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:28:00.294649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:28:00.294649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:00.294649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:00.294649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:00.294649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:00.294649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:00.295852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:00.295852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:00.295852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:00.295852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:00.295852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:00.295852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:00.295852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:28:00.295852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:28:00.295852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:28:00.295852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:28:00.295852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:00.295852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:00.295852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:00.295852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:00.295852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:00.295852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:00.296714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:00.296714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:00.296714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:00.296714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:00.296714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:00.296714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:00.296714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:00.296714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:00.296714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:00.296714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:00.296714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:28:00.296714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:28:00.296714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 18:28:00.296714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:28:00.298816 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 519/1014] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:28:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:28:00.302000 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f02513-783f-450c-acd8-8e8f69c421c3 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:00.414019 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f02513-783f-450c-acd8-8e8f69c421c3 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:00.448689 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1dc17f-10e4-4216-b1dd-b65e8c8d7963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:00.449351 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dc17f-10e4-4216-b1dd-b65e8c8d7963 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:00.449637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dc17f-10e4-4216-b1dd-b65e8c8d7963 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:00.449918 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1dc17f-10e4-4216-b1dd-b65e8c8d7963 None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:00.450667 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1dc17f-10e4-4216-b1dd-b65e8c8d7963 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:00.450975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1dc17f-10e4-4216-b1dd-b65e8c8d7963 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:00.452338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1dc17f-10e4-4216-b1dd-b65e8c8d7963 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:00.462318 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 496/1015] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Jun 27 18:28:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:00.463405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f1652-075b-4b45-a2df-4153351bcd45 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:00.569145 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f1652-075b-4b45-a2df-4153351bcd45 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:00.572916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:00.603103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47193e-0d89-41a6-898a-6dbbc7f34aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:00.604272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47193e-0d89-41a6-898a-6dbbc7f34aca None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:00.604908 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47193e-0d89-41a6-898a-6dbbc7f34aca None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:00.605665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47193e-0d89-41a6-898a-6dbbc7f34aca None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:00.606659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c47193e-0d89-41a6-898a-6dbbc7f34aca None admin] RBAC: Adding query filter params (type=compute) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:00.606964 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c47193e-0d89-41a6-898a-6dbbc7f34aca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:00.608316 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c47193e-0d89-41a6-898a-6dbbc7f34aca None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:00.613467 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 520/1016] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Jun 27 18:28:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:28:00.619365 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1dc17f-10e4-4216-b1dd-b65e8c8d7963 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:00.635390 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:00.721117 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1dc17f-10e4-4216-b1dd-b65e8c8d7963 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:00.759358 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53f1f7-8b36-457e-a188-ea5084b9f447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:00.760044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53f1f7-8b36-457e-a188-ea5084b9f447 None admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:00.760288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53f1f7-8b36-457e-a188-ea5084b9f447 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:00.760558 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53f1f7-8b36-457e-a188-ea5084b9f447 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:00.761292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53f1f7-8b36-457e-a188-ea5084b9f447 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:00.762660 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53f1f7-8b36-457e-a188-ea5084b9f447 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:00.873613 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 497/1017] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Jun 27 18:28:00 2022] POST /identity/v3/endpoints => generated 334 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:28:01.144321 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c9de9-5eee-40b9-8010-75b4cc4cfe50 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:01.144831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c9de9-5eee-40b9-8010-75b4cc4cfe50 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:01.144831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c9de9-5eee-40b9-8010-75b4cc4cfe50 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:01.146327 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 521/1018] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:28:01.151396 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066066c-8299-4040-8acb-77821a205205 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:01.151693 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066066c-8299-4040-8acb-77821a205205 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:01.151977 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066066c-8299-4040-8acb-77821a205205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:01.169412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-8066066c-8299-4040-8acb-77821a205205 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:01.170641 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-8066066c-8299-4040-8acb-77821a205205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:01.235504 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:01.247380 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 498/1019] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:01 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:28:01.270157 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c9de9-5eee-40b9-8010-75b4cc4cfe50 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:01.376876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c9de9-5eee-40b9-8010-75b4cc4cfe50 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:01.412143 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32314df-76a5-4a75-8bab-c6327ec21629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:01.412715 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32314df-76a5-4a75-8bab-c6327ec21629 None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:01.412993 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32314df-76a5-4a75-8bab-c6327ec21629 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:01.413263 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32314df-76a5-4a75-8bab-c6327ec21629 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:01.414043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32314df-76a5-4a75-8bab-c6327ec21629 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:01.415421 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32314df-76a5-4a75-8bab-c6327ec21629 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:01.519366 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 522/1020] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Jun 27 18:28:01 2022] POST /identity/v3/services => generated 242 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:28:02.313643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d155e-3395-40c0-90d5-234765f443ce None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:02.314191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d155e-3395-40c0-90d5-234765f443ce None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:02.314191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d155e-3395-40c0-90d5-234765f443ce None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:02.315629 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 499/1021] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:28:02.320697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e12ca-e735-48ca-801d-f8cf59c312cb None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:02.321576 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e12ca-e735-48ca-801d-f8cf59c312cb None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:02.324968 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e12ca-e735-48ca-801d-f8cf59c312cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:02.338453 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-fc9e12ca-e735-48ca-801d-f8cf59c312cb None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:02.339636 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-fc9e12ca-e735-48ca-801d-f8cf59c312cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:02.362950 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:02.412905 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 523/1022] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:28:02.438829 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d155e-3395-40c0-90d5-234765f443ce None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:02.539442 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d155e-3395-40c0-90d5-234765f443ce None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:02.573866 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d196f6e8-f48a-4e08-a630-38457bc27220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:02.574606 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196f6e8-f48a-4e08-a630-38457bc27220 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:02.574888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196f6e8-f48a-4e08-a630-38457bc27220 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:02.575160 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196f6e8-f48a-4e08-a630-38457bc27220 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:02.576331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d196f6e8-f48a-4e08-a630-38457bc27220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:02.577983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d196f6e8-f48a-4e08-a630-38457bc27220 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:02.585612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 500/1023] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:28:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:02.585758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e12ca-e735-48ca-801d-f8cf59c312cb None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:02.695520 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e12ca-e735-48ca-801d-f8cf59c312cb None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:02.731574 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd731d3-9b2a-48f9-94cf-a2f353491bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:02.732231 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd731d3-9b2a-48f9-94cf-a2f353491bc5 None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:02.732511 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd731d3-9b2a-48f9-94cf-a2f353491bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:02.732784 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd731d3-9b2a-48f9-94cf-a2f353491bc5 None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:02.733480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd731d3-9b2a-48f9-94cf-a2f353491bc5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:02.739156 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd731d3-9b2a-48f9-94cf-a2f353491bc5 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:02.858419 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 524/1024] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Jun 27 18:28:02 2022] POST /identity/v3/users => generated 285 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:28:03.022055 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abfd98-19a5-4ec8-9e39-70a5c734962b None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:03.022343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abfd98-19a5-4ec8-9e39-70a5c734962b None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:03.022677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abfd98-19a5-4ec8-9e39-70a5c734962b None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:03.024668 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 501/1025] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:28:03.030021 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e72fb7-3185-41c5-87c2-7b510f015ea6 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:03.030313 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e72fb7-3185-41c5-87c2-7b510f015ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:03.030621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e72fb7-3185-41c5-87c2-7b510f015ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:03.048185 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-a8e72fb7-3185-41c5-87c2-7b510f015ea6 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:03.049365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-a8e72fb7-3185-41c5-87c2-7b510f015ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:03.052353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:03.125533 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 525/1026] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:28:03.150486 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8abfd98-19a5-4ec8-9e39-70a5c734962b None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:03.218549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:03.262980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8abfd98-19a5-4ec8-9e39-70a5c734962b None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:03.300917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f1164-c7a4-4977-8309-b9758d867631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:03.301716 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f1164-c7a4-4977-8309-b9758d867631 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:03.301991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f1164-c7a4-4977-8309-b9758d867631 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:03.302256 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f1164-c7a4-4977-8309-b9758d867631 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:03.313378 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f1164-c7a4-4977-8309-b9758d867631 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:28:03.322740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-4f2f1164-c7a4-4977-8309-b9758d867631 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 18:28:03.322740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:03.322740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:03.322740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:03.322740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:03.322740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:03.322740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:03.322740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:03.322740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:03.322740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:03.322740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:03.322740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:03.322740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:28:03.322740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:28:03.322740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:28:03.322740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:28:03.324197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:28:03.324197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:28:03.324197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:28:03.324197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:28:03.324197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:03.324197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:03.324197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:03.324197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:03.324197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:03.324197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:03.324197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:03.324197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:03.324197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:03.324197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:03.324197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:03.324197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:03.325594 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:03.325594 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:03.325594 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:03.325594 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:03.325594 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:28:03.325594 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:28:03.325594 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:28:03.325594 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:28:03.325594 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:28:03.325594 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:28:03.325594 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 18:28:03.325594 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:28:03.326476 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 502/1027] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Jun 27 18:28:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:28:03.330047 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e72fb7-3185-41c5-87c2-7b510f015ea6 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:03.435333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e72fb7-3185-41c5-87c2-7b510f015ea6 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:03.469402 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadfe4b-2427-4559-a95e-c1a3488a48e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:03.470062 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadfe4b-2427-4559-a95e-c1a3488a48e0 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:03.470349 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadfe4b-2427-4559-a95e-c1a3488a48e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:03.470651 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadfe4b-2427-4559-a95e-c1a3488a48e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:03.471394 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadfe4b-2427-4559-a95e-c1a3488a48e0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:03.471699 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadfe4b-2427-4559-a95e-c1a3488a48e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:03.473061 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadfe4b-2427-4559-a95e-c1a3488a48e0 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:03.486029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 526/1028] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Jun 27 18:28:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:03.487622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f1164-c7a4-4977-8309-b9758d867631 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:03.571123 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c97d1-9104-46d8-b800-a07093eb7e25 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:03.571829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c97d1-9104-46d8-b800-a07093eb7e25 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:03.571922 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c97d1-9104-46d8-b800-a07093eb7e25 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:03.573359 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 527/1029] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:28:03.578187 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d43df3-3ac1-4676-bd94-d9048df0ff45 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:03.578496 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d43df3-3ac1-4676-bd94-d9048df0ff45 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:03.578779 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d43df3-3ac1-4676-bd94-d9048df0ff45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:03.583996 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:03.591522 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f1164-c7a4-4977-8309-b9758d867631 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:03.596438 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-98d43df3-3ac1-4676-bd94-d9048df0ff45 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:03.605538 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-98d43df3-3ac1-4676-bd94-d9048df0ff45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:03.616060 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:03.627974 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c436e6-b4fa-430c-b814-f3cc3da8bf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:03.628758 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c436e6-b4fa-430c-b814-f3cc3da8bf02 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:03.629039 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c436e6-b4fa-430c-b814-f3cc3da8bf02 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:03.629331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c436e6-b4fa-430c-b814-f3cc3da8bf02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:03.641038 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c436e6-b4fa-430c-b814-f3cc3da8bf02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:03.642510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c436e6-b4fa-430c-b814-f3cc3da8bf02 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:03.660119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-d2c436e6-b4fa-430c-b814-f3cc3da8bf02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:28:03.660119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:03.660119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:03.660119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:03.660119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:03.660119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:03.660119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:03.660119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:03.660119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:03.660119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:03.660119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:03.660119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:03.660119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:28:03.660119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:28:03.660119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:28:03.661043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:28:03.661043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:03.661043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:03.661043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:03.661043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:03.661043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:03.661043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:03.661043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:03.661043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:03.661043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:03.661043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:03.661043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:03.661043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:03.661043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:03.661043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:03.661043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:03.661043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:03.661929 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:28:03.661929 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:28:03.661929 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:28:03.661929 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:28:03.661929 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 503/1030] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:28:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:28:03.667537 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c436e6-b4fa-430c-b814-f3cc3da8bf02 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:03.678441 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 528/1031] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 18:28:03.698079 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d43df3-3ac1-4676-bd94-d9048df0ff45 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:03.780332 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c436e6-b4fa-430c-b814-f3cc3da8bf02 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:03.815767 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ddb95f-a679-46bc-b2be-9f647dd10a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:03.816420 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ddb95f-a679-46bc-b2be-9f647dd10a6b None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:03.816701 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ddb95f-a679-46bc-b2be-9f647dd10a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:03.816978 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ddb95f-a679-46bc-b2be-9f647dd10a6b None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:03.817696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ddb95f-a679-46bc-b2be-9f647dd10a6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:03.817950 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d43df3-3ac1-4676-bd94-d9048df0ff45 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:03.818055 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ddb95f-a679-46bc-b2be-9f647dd10a6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:03.819377 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ddb95f-a679-46bc-b2be-9f647dd10a6b None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:03.831816 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 504/1032] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:28:03.837463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ddb95f-a679-46bc-b2be-9f647dd10a6b None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:03.854279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:03.861003 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f8d61-c538-431a-b861-04d43aeebe16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:03.862246 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f8d61-c538-431a-b861-04d43aeebe16 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:03.862773 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f8d61-c538-431a-b861-04d43aeebe16 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:03.863962 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f8d61-c538-431a-b861-04d43aeebe16 None admin] PATH_INFO: `/v3/services/image` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:03.864741 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f8d61-c538-431a-b861-04d43aeebe16 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:03.866856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f8d61-c538-431a-b861-04d43aeebe16 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:03.873717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:03.881396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-4f3f8d61-c538-431a-b861-04d43aeebe16 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 18:28:03.881396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:03.881396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:28:03.881396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:28:03.881396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:28:03.881396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:28:03.881396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:28:03.881396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:28:03.881396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 18:28:03.881396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:28:03.881396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:28:03.881396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:28:03.881396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:03.881396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:03.881396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:03.881396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:03.881396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:03.882423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:03.882423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:03.882423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:03.882423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:03.882423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:03.882423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:03.882423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:28:03.882423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:28:03.882423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:28:03.882423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:28:03.882423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:03.882423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:03.882423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:03.882423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:03.882423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:03.882423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:03.883286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:03.883286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:03.883286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:03.883286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:03.883286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:03.883286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:03.883286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:03.883286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:03.883286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:03.883286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:03.883286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:28:03.883286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:28:03.883286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 18:28:03.883286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:28:03.883286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 529/1033] 10.222.0.15 () {56 vars in 1105 bytes} [Mon Jun 27 18:28:03 2022] GET /identity/v3/services/image => generated 86 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:28:03.889669 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f8d61-c538-431a-b861-04d43aeebe16 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:03.947659 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ddb95f-a679-46bc-b2be-9f647dd10a6b None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:03.984927 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a044e6-f97e-42eb-9531-9511be26a846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:03.985685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a044e6-f97e-42eb-9531-9511be26a846 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:03.985978 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a044e6-f97e-42eb-9531-9511be26a846 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:03.986254 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a044e6-f97e-42eb-9531-9511be26a846 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:03.992887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f8d61-c538-431a-b861-04d43aeebe16 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:03.993873 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a044e6-f97e-42eb-9531-9511be26a846 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:03.995932 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a044e6-f97e-42eb-9531-9511be26a846 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:04.010317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-d0a044e6-f97e-42eb-9531-9511be26a846 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 18:28:04.010317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:04.010317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:04.010317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:04.010317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:04.010317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:04.010317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:04.010317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:04.010317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:04.010317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:04.010317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:04.010317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:04.010317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:28:04.010317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:28:04.010317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:28:04.010317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:28:04.011369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:04.011369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:04.011369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:28:04.011369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:28:04.011369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:28:04.011369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:28:04.011369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:04.011369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:04.011369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:04.011369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:04.011369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:04.011369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:04.011369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:04.011369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:04.011369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:04.011369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:04.011369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:04.011369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:04.012328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:04.012328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:04.012328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:28:04.012328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:28:04.012328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:28:04.012328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:28:04.012328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:28:04.012328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:28:04.012328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 18:28:04.012328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:28:04.012328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 505/1034] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Jun 27 18:28:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:28:04.017134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a044e6-f97e-42eb-9531-9511be26a846 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:04.033191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc95755-1b42-49c7-806e-53cc372f5d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:04.033885 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc95755-1b42-49c7-806e-53cc372f5d9c None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:04.034177 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc95755-1b42-49c7-806e-53cc372f5d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:04.034487 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc95755-1b42-49c7-806e-53cc372f5d9c None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:04.035204 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc95755-1b42-49c7-806e-53cc372f5d9c None admin] RBAC: Adding query filter params (name=image) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:04.035512 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc95755-1b42-49c7-806e-53cc372f5d9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:04.036889 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc95755-1b42-49c7-806e-53cc372f5d9c None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:04.047268 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 530/1035] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:28:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:28:04.048325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc95755-1b42-49c7-806e-53cc372f5d9c None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:04.114270 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:04.118263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:04.124752 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a044e6-f97e-42eb-9531-9511be26a846 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:04.156638 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc95755-1b42-49c7-806e-53cc372f5d9c None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:04.158940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b32dc-ac94-4c93-9f66-ed3c8dd2b778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:04.159630 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b32dc-ac94-4c93-9f66-ed3c8dd2b778 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:04.159910 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b32dc-ac94-4c93-9f66-ed3c8dd2b778 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:04.160186 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b32dc-ac94-4c93-9f66-ed3c8dd2b778 None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:04.161036 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b32dc-ac94-4c93-9f66-ed3c8dd2b778 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:04.161368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b32dc-ac94-4c93-9f66-ed3c8dd2b778 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:04.162841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b32dc-ac94-4c93-9f66-ed3c8dd2b778 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:04.181630 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 506/1036] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Jun 27 18:28:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:28:04.187155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b32dc-ac94-4c93-9f66-ed3c8dd2b778 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:04.199098 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6810ba-04bb-444f-91a6-65aabe74d9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:04.200032 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6810ba-04bb-444f-91a6-65aabe74d9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:04.200312 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6810ba-04bb-444f-91a6-65aabe74d9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:04.200578 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6810ba-04bb-444f-91a6-65aabe74d9f6 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:04.201281 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6810ba-04bb-444f-91a6-65aabe74d9f6 None admin] RBAC: Adding query filter params (type=image) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:04.201582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6810ba-04bb-444f-91a6-65aabe74d9f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:04.203005 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6810ba-04bb-444f-91a6-65aabe74d9f6 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:04.208112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 531/1037] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:28:04 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:28:04.218346 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6810ba-04bb-444f-91a6-65aabe74d9f6 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:04.299691 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b32dc-ac94-4c93-9f66-ed3c8dd2b778 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:04.325103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6810ba-04bb-444f-91a6-65aabe74d9f6 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:04.335878 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7b200-a09b-4624-8faa-bb9a11b5feb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:04.337326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7b200-a09b-4624-8faa-bb9a11b5feb2 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:04.337604 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7b200-a09b-4624-8faa-bb9a11b5feb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:04.338010 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7b200-a09b-4624-8faa-bb9a11b5feb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:04.350794 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:04.361492 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9ac0e-11bb-4016-b025-ef5300c18c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:04.361970 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:04.362243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9ac0e-11bb-4016-b025-ef5300c18c9f None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:04.362567 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9ac0e-11bb-4016-b025-ef5300c18c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:04.362846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9ac0e-11bb-4016-b025-ef5300c18c9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:04.363715 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf9ac0e-11bb-4016-b025-ef5300c18c9f None admin] RBAC: Adding query filter params (interface=public, service_id=deee1e7c6e2c4251a9d88b7bd077f14d, region_id=RegionOne) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:04.364054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf9ac0e-11bb-4016-b025-ef5300c18c9f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:04.365427 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf9ac0e-11bb-4016-b025-ef5300c18c9f None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:04.371143 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 532/1038] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Jun 27 18:28:04 2022] GET /identity/v3/endpoints?service_id=deee1e7c6e2c4251a9d88b7bd077f14d&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:28:04.376630 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9ac0e-11bb-4016-b025-ef5300c18c9f None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:04.413466 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7b200-a09b-4624-8faa-bb9a11b5feb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:04.415123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7b200-a09b-4624-8faa-bb9a11b5feb2 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:04.491048 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9ac0e-11bb-4016-b025-ef5300c18c9f None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:04.526679 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 507/1039] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:28:04 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 18:28:04.527451 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7b200-a09b-4624-8faa-bb9a11b5feb2 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:04.528998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce7c08-dc86-43f9-b7f0-910e456848ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:04.529669 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce7c08-dc86-43f9-b7f0-910e456848ee None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:04.529952 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce7c08-dc86-43f9-b7f0-910e456848ee None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:04.530284 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce7c08-dc86-43f9-b7f0-910e456848ee None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:04.531027 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce7c08-dc86-43f9-b7f0-910e456848ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:04.532447 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce7c08-dc86-43f9-b7f0-910e456848ee None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:04.539503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 533/1040] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Jun 27 18:28:04 2022] GET /identity/v3/services => generated 1957 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:28:04.598750 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:04.651843 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7b200-a09b-4624-8faa-bb9a11b5feb2 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:04.687828 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a998105-5d6b-4782-8a3c-7d1dd097f3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:04.688615 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a998105-5d6b-4782-8a3c-7d1dd097f3ee None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:04.688894 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a998105-5d6b-4782-8a3c-7d1dd097f3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:04.689165 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a998105-5d6b-4782-8a3c-7d1dd097f3ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:04.702596 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a998105-5d6b-4782-8a3c-7d1dd097f3ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:04.704018 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a998105-5d6b-4782-8a3c-7d1dd097f3ee None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:04.721911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-4a998105-5d6b-4782-8a3c-7d1dd097f3ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:28:04.721911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:04.721911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:04.721911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:04.721911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:04.721911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:04.721911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:04.721911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:04.721911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:04.721911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:04.721911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:04.721911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:04.721911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:28:04.721911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:28:04.721911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:28:04.722859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:28:04.722859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:04.722859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:04.722859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:04.722859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:04.722859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:04.722859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:04.722859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:04.722859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:04.722859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:04.722859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:04.722859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:04.722859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:04.722859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:04.722859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:04.722859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:04.722859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:04.723795 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:28:04.723795 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:28:04.723795 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:28:04.723795 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:28:04.723795 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 508/1041] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:28:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:28:04.728992 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce7c08-dc86-43f9-b7f0-910e456848ee None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:04.791687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:04.839619 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce7c08-dc86-43f9-b7f0-910e456848ee None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:04.875589 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e525dd-9736-473e-91be-f3cbabfc0878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:04.876162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e525dd-9736-473e-91be-f3cbabfc0878 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:04.876442 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e525dd-9736-473e-91be-f3cbabfc0878 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:04.876715 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e525dd-9736-473e-91be-f3cbabfc0878 None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:04.877435 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e525dd-9736-473e-91be-f3cbabfc0878 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:04.877737 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e525dd-9736-473e-91be-f3cbabfc0878 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:04.879119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e525dd-9736-473e-91be-f3cbabfc0878 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:04.895703 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 534/1042] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:28:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:04.897316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a998105-5d6b-4782-8a3c-7d1dd097f3ee None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:04.996943 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85fdbd-2acb-46d2-859a-17d8edee2fc0 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:04.997272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85fdbd-2acb-46d2-859a-17d8edee2fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:04.997511 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85fdbd-2acb-46d2-859a-17d8edee2fc0 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:04.999044 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 535/1043] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:28:05.003709 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6991f-d902-4b04-b922-3983a872f00e None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:05.004560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6991f-d902-4b04-b922-3983a872f00e None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:05.004560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6991f-d902-4b04-b922-3983a872f00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:05.006855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a998105-5d6b-4782-8a3c-7d1dd097f3ee None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:05.009752 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:05.022509 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-18c6991f-d902-4b04-b922-3983a872f00e None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:05.023685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-18c6991f-d902-4b04-b922-3983a872f00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:05.043079 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6ca0c-b210-4f08-b69d-a454a45ea100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:05.043872 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6ca0c-b210-4f08-b69d-a454a45ea100 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:05.044148 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6ca0c-b210-4f08-b69d-a454a45ea100 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:05.044419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6ca0c-b210-4f08-b69d-a454a45ea100 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:05.051527 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b6ca0c-b210-4f08-b69d-a454a45ea100 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:05.053462 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b6ca0c-b210-4f08-b69d-a454a45ea100 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:05.067687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-b8b6ca0c-b210-4f08-b69d-a454a45ea100 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:28:05.067687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:05.067687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:05.067687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:05.067687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:05.067687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:05.067687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:05.067687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:05.067687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:05.067687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:05.067687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:05.067687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:05.067687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:28:05.067687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:28:05.067687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:28:05.068652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:28:05.068652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:05.068652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:05.068652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:05.068652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:05.068652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:05.068652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:05.068652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:05.068652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:05.068652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:05.068652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:05.068652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:05.068652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:05.068652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:05.068652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:05.068652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:05.068652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:05.069572 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:28:05.069572 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:28:05.069572 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:28:05.069572 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:28:05.069572 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:28:05.069572 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:28:05.069572 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:28:05.069572 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:28:05.069572 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 509/1044] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:28:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:28:05.076283 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6ca0c-b210-4f08-b69d-a454a45ea100 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:05.098507 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 536/1045] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 18:28:05.122373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6991f-d902-4b04-b922-3983a872f00e None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:05.172345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:05.186836 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6ca0c-b210-4f08-b69d-a454a45ea100 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:05.223923 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb2e60-c084-4171-8734-87cf3444851c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:05.224573 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb2e60-c084-4171-8734-87cf3444851c None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:05.224853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb2e60-c084-4171-8734-87cf3444851c None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:05.225124 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb2e60-c084-4171-8734-87cf3444851c None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:05.225904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb2e60-c084-4171-8734-87cf3444851c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:05.226209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb2e60-c084-4171-8734-87cf3444851c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:05.228434 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb2e60-c084-4171-8734-87cf3444851c None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:05.233099 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6991f-d902-4b04-b922-3983a872f00e None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:05.241345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 510/1046] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:28:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:05.247528 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb2e60-c084-4171-8734-87cf3444851c None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:05.268386 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c7e4c-57b0-48a2-abcd-50571e787c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:05.269189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c7e4c-57b0-48a2-abcd-50571e787c8c None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:05.269459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c7e4c-57b0-48a2-abcd-50571e787c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:05.269733 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c7e4c-57b0-48a2-abcd-50571e787c8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:05.274597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:05.280811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c7e4c-57b0-48a2-abcd-50571e787c8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:28:05.288572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-e26c7e4c-57b0-48a2-abcd-50571e787c8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:28:05.288572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:05.288572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:05.288572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:05.288572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:05.288572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:05.288572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:05.288572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:05.288572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:05.288572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:05.288572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:05.288572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:05.288572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:28:05.288572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:28:05.288572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:28:05.288572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:28:05.289495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:28:05.289495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:28:05.289495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:28:05.289495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:28:05.289495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:05.289495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:05.289495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:05.289495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:05.289495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:05.289495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:05.289495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:05.289495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:05.289495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:05.289495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:05.289495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:05.289495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:05.291741 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:05.291741 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:05.291741 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:05.291741 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:05.291741 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:28:05.291741 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:28:05.291741 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:28:05.291741 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:28:05.291741 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:28:05.291741 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:28:05.291741 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:28:05.291741 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:28:05.294728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 537/1047] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:28:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:28:05.296147 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c7e4c-57b0-48a2-abcd-50571e787c8c None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:05.354779 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb2e60-c084-4171-8734-87cf3444851c None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:05.391680 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1df9f-d607-47db-a0e6-b6d092874002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:05.392448 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1df9f-d607-47db-a0e6-b6d092874002 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:05.392727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1df9f-d607-47db-a0e6-b6d092874002 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:05.392998 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1df9f-d607-47db-a0e6-b6d092874002 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:05.393831 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa1df9f-d607-47db-a0e6-b6d092874002 None admin] RBAC: Adding query filter params (role.id=3ef93c53540740d6b89875c0ff507fab, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=1dddd4efb44c4152bb4151f7715f2223) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:05.394120 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa1df9f-d607-47db-a0e6-b6d092874002 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:05.395566 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa1df9f-d607-47db-a0e6-b6d092874002 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:05.410424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 511/1048] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:28:05 2022] GET /identity/v3/role_assignments?user.id=1dddd4efb44c4152bb4151f7715f2223&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=3ef93c53540740d6b89875c0ff507fab => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:28:05.410640 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c7e4c-57b0-48a2-abcd-50571e787c8c None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:05.410640 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:05.444437 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31cd548-7c45-4647-a48a-b64927830d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:05.445024 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cd548-7c45-4647-a48a-b64927830d78 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:05.445309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cd548-7c45-4647-a48a-b64927830d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:05.445584 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cd548-7c45-4647-a48a-b64927830d78 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:05.446933 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31cd548-7c45-4647-a48a-b64927830d78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:05.447337 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31cd548-7c45-4647-a48a-b64927830d78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:05.448816 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31cd548-7c45-4647-a48a-b64927830d78 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:05.464506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 538/1049] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:28:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:28:05.465926 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1df9f-d607-47db-a0e6-b6d092874002 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:05.563055 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:05.575608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1df9f-d607-47db-a0e6-b6d092874002 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:05.610878 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19004bec-41a0-4639-b6de-09f214dbc357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:05.611751 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19004bec-41a0-4639-b6de-09f214dbc357 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:05.612032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19004bec-41a0-4639-b6de-09f214dbc357 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:05.612310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19004bec-41a0-4639-b6de-09f214dbc357 None admin] PATH_INFO: `/v3/users/2a7861ee7e4648dfa8be38b451b559f7` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:05.625811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19004bec-41a0-4639-b6de-09f214dbc357 None admin] RBAC: Authorizing `identity:get_user(user_id=2a7861ee7e4648dfa8be38b451b559f7)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:05.627275 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19004bec-41a0-4639-b6de-09f214dbc357 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:05.634276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 512/1050] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:28:05 2022] GET /identity/v3/users/2a7861ee7e4648dfa8be38b451b559f7 => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:05.635285 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31cd548-7c45-4647-a48a-b64927830d78 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:05.750033 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31cd548-7c45-4647-a48a-b64927830d78 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:05.785667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bfd64a-a0fb-4562-a859-195611e4256a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:05.786309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfd64a-a0fb-4562-a859-195611e4256a None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:05.786698 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfd64a-a0fb-4562-a859-195611e4256a None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:05.786978 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bfd64a-a0fb-4562-a859-195611e4256a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:05.818970 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:05.856791 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bfd64a-a0fb-4562-a859-195611e4256a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:05.858244 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bfd64a-a0fb-4562-a859-195611e4256a None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:05.961282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 539/1051] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:28:05 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 18:28:05.963086 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19004bec-41a0-4639-b6de-09f214dbc357 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:06.067407 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19004bec-41a0-4639-b6de-09f214dbc357 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:06.103450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0226f-eef6-4746-b2ac-fa1543ce87c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:06.104304 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0226f-eef6-4746-b2ac-fa1543ce87c4 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:06.104580 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0226f-eef6-4746-b2ac-fa1543ce87c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:06.104848 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0226f-eef6-4746-b2ac-fa1543ce87c4 None admin] PATH_INFO: `/v3/projects/9fa2d394f79f4e5592f517d8a0f2b340` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:06.106290 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0226f-eef6-4746-b2ac-fa1543ce87c4 None admin] RBAC: Authorizing `identity:get_project(project_id=9fa2d394f79f4e5592f517d8a0f2b340)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:06.107745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0226f-eef6-4746-b2ac-fa1543ce87c4 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:06.114287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 513/1052] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:28:05 2022] GET /identity/v3/projects/9fa2d394f79f4e5592f517d8a0f2b340 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:06.115454 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bfd64a-a0fb-4562-a859-195611e4256a None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:06.181660 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:06.219114 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bfd64a-a0fb-4562-a859-195611e4256a None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:06.253786 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3cb9d-d83b-4b17-bc84-132acf7f84a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:06.254449 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3cb9d-d83b-4b17-bc84-132acf7f84a4 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:06.254738 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3cb9d-d83b-4b17-bc84-132acf7f84a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:06.255092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3cb9d-d83b-4b17-bc84-132acf7f84a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:06.255929 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f3cb9d-d83b-4b17-bc84-132acf7f84a4 None admin] RBAC: Adding query filter params (role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f, scope.project.id=9fa2d394f79f4e5592f517d8a0f2b340, user.id=2a7861ee7e4648dfa8be38b451b559f7) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:06.256243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f3cb9d-d83b-4b17-bc84-132acf7f84a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:06.257673 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f3cb9d-d83b-4b17-bc84-132acf7f84a4 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:06.270414 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 540/1053] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:28:06 2022] GET /identity/v3/role_assignments?user.id=2a7861ee7e4648dfa8be38b451b559f7&scope.project.id=9fa2d394f79f4e5592f517d8a0f2b340&role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:28:06.591615 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e8701-327d-49b8-ae0e-ec28462604d5 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:06.592184 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e8701-327d-49b8-ae0e-ec28462604d5 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:06.592184 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e8701-327d-49b8-ae0e-ec28462604d5 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:06.594001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 514/1054] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:28:06.599008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602c471-16e0-486a-aa50-b6528e024cd6 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:06.599402 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602c471-16e0-486a-aa50-b6528e024cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:06.599564 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602c471-16e0-486a-aa50-b6528e024cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:06.616975 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-c602c471-16e0-486a-aa50-b6528e024cd6 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:06.618184 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c602c471-16e0-486a-aa50-b6528e024cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:06.685584 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 541/1055] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:28:06.708053 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e8701-327d-49b8-ae0e-ec28462604d5 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:06.724656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:06.819727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e8701-327d-49b8-ae0e-ec28462604d5 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:06.854681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74058a2c-1151-4fdf-8f1d-bbed9e127032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:06.855388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74058a2c-1151-4fdf-8f1d-bbed9e127032 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:06.855669 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74058a2c-1151-4fdf-8f1d-bbed9e127032 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:06.856202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74058a2c-1151-4fdf-8f1d-bbed9e127032 None admin] PATH_INFO: `/v3/services/image` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:06.856847 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74058a2c-1151-4fdf-8f1d-bbed9e127032 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:06.858321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74058a2c-1151-4fdf-8f1d-bbed9e127032 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:06.870071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-74058a2c-1151-4fdf-8f1d-bbed9e127032 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 18:28:06.870071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:06.870071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:28:06.870071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:28:06.870071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:28:06.870071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:28:06.870071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:28:06.870071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:28:06.870071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 18:28:06.870071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:28:06.870071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:28:06.870071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:28:06.870071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:06.870071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:06.870071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:06.870071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:06.870071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:06.871257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:06.871257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:06.871257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:06.871257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:06.871257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:06.871257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:06.871257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:28:06.871257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:28:06.871257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:28:06.871257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:28:06.871257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:06.871257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:06.871257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:06.871257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:06.871257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:06.871257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:06.872106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:06.872106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:06.872106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:06.872106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:06.872106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:06.872106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:06.872106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:06.872106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:06.872106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:06.872106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:06.872106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:28:06.872106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:28:06.872106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 18:28:06.872106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:28:06.876367 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 515/1056] 10.222.0.15 () {56 vars in 1105 bytes} [Mon Jun 27 18:28:06 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:28:06.877205 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602c471-16e0-486a-aa50-b6528e024cd6 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:06.935565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:06.977761 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602c471-16e0-486a-aa50-b6528e024cd6 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:07.012778 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494e2e3-0215-433f-b973-adf9f8530228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:07.013436 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494e2e3-0215-433f-b973-adf9f8530228 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:07.013764 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494e2e3-0215-433f-b973-adf9f8530228 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:07.014043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494e2e3-0215-433f-b973-adf9f8530228 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:07.014795 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494e2e3-0215-433f-b973-adf9f8530228 None admin] RBAC: Adding query filter params (name=image) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:07.015102 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494e2e3-0215-433f-b973-adf9f8530228 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:07.016466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494e2e3-0215-433f-b973-adf9f8530228 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:07.026145 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 542/1057] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:28:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:28:07.027103 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74058a2c-1151-4fdf-8f1d-bbed9e127032 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:07.107153 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:07.128890 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74058a2c-1151-4fdf-8f1d-bbed9e127032 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:07.164865 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1459ad7-7bb0-427f-95c2-fa9ff8732564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:07.165517 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1459ad7-7bb0-427f-95c2-fa9ff8732564 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:07.165801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1459ad7-7bb0-427f-95c2-fa9ff8732564 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:07.166079 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1459ad7-7bb0-427f-95c2-fa9ff8732564 None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:07.166822 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1459ad7-7bb0-427f-95c2-fa9ff8732564 None admin] RBAC: Adding query filter params (type=image) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:07.167130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1459ad7-7bb0-427f-95c2-fa9ff8732564 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:07.168491 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1459ad7-7bb0-427f-95c2-fa9ff8732564 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:07.173622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 516/1058] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:28:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:28:07.179061 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494e2e3-0215-433f-b973-adf9f8530228 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:07.281502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494e2e3-0215-433f-b973-adf9f8530228 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:07.305836 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:07.317830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e35f3-d8f1-44d8-b9ca-2e642bdad21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:07.318516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e35f3-d8f1-44d8-b9ca-2e642bdad21c None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:07.318804 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e35f3-d8f1-44d8-b9ca-2e642bdad21c None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:07.319151 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e35f3-d8f1-44d8-b9ca-2e642bdad21c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:07.319990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e35f3-d8f1-44d8-b9ca-2e642bdad21c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:07.321492 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e35f3-d8f1-44d8-b9ca-2e642bdad21c None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:07.398492 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 543/1059] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Jun 27 18:28:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:28:07.459864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e127b96-7d8c-4909-850d-c99ca1000baa None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:07.460149 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e127b96-7d8c-4909-850d-c99ca1000baa None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:07.460427 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e127b96-7d8c-4909-850d-c99ca1000baa None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:07.462960 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 517/1060] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:28:07.468696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e330972-9b52-4002-ac3f-f8983af40229 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:07.468984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e330972-9b52-4002-ac3f-f8983af40229 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:07.469264 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e330972-9b52-4002-ac3f-f8983af40229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:07.487500 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-9e330972-9b52-4002-ac3f-f8983af40229 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:07.488693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-9e330972-9b52-4002-ac3f-f8983af40229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:07.566111 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 544/1061] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:28:07.589339 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e127b96-7d8c-4909-850d-c99ca1000baa None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:07.698007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e127b96-7d8c-4909-850d-c99ca1000baa None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:07.733225 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581cece-e6d6-4ed0-b200-6cad5f5181d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:07.734409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581cece-e6d6-4ed0-b200-6cad5f5181d4 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:07.734893 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581cece-e6d6-4ed0-b200-6cad5f5181d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:07.735392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581cece-e6d6-4ed0-b200-6cad5f5181d4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:07.749565 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581cece-e6d6-4ed0-b200-6cad5f5181d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:28:07.757717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-8581cece-e6d6-4ed0-b200-6cad5f5181d4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 18:28:07.757717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:07.757717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:07.757717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:07.757717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:07.757717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:07.757717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:07.757717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:07.757717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:07.757717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:07.757717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:07.757717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:07.757717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:28:07.757717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:28:07.757717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:28:07.757717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:28:07.758904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:28:07.758904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:28:07.758904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:28:07.758904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:28:07.758904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:07.758904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:07.758904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:07.758904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:07.758904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:07.758904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:07.758904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:07.758904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:07.758904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:07.758904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:07.758904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:07.758904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:07.760001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:07.760001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:07.760001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:07.760001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:07.760001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:28:07.760001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:28:07.760001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:28:07.760001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:28:07.760001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:28:07.760001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:28:07.760001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 18:28:07.760001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:28:07.760001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 518/1062] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Jun 27 18:28:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:28:07.767079 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e330972-9b52-4002-ac3f-f8983af40229 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:07.867360 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e330972-9b52-4002-ac3f-f8983af40229 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:07.869844 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:07.904140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4821663-0e7d-4609-adff-6b726f7f439a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:07.904809 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4821663-0e7d-4609-adff-6b726f7f439a None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:07.905091 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4821663-0e7d-4609-adff-6b726f7f439a None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:07.905364 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4821663-0e7d-4609-adff-6b726f7f439a None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:07.906109 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4821663-0e7d-4609-adff-6b726f7f439a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:07.906441 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4821663-0e7d-4609-adff-6b726f7f439a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:07.907813 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4821663-0e7d-4609-adff-6b726f7f439a None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:07.920542 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 545/1063] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Jun 27 18:28:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:07.921964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581cece-e6d6-4ed0-b200-6cad5f5181d4 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:07.925082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:08.024446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581cece-e6d6-4ed0-b200-6cad5f5181d4 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:08.057478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff15fe-2360-4721-bdf4-c37449369356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:08.058357 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff15fe-2360-4721-bdf4-c37449369356 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:08.058668 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff15fe-2360-4721-bdf4-c37449369356 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:08.058942 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff15fe-2360-4721-bdf4-c37449369356 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:08.069572 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff15fe-2360-4721-bdf4-c37449369356 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:08.071114 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff15fe-2360-4721-bdf4-c37449369356 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:08.088392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-32ff15fe-2360-4721-bdf4-c37449369356 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:28:08.088392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:08.088392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:08.088392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:08.088392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:08.088392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:08.088392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:08.088392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:08.088392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:08.088392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:08.088392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:08.088392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:08.088392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:28:08.088392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:28:08.088392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:28:08.089292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:28:08.089292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:08.089292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:08.089292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:08.089292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:08.089292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:08.089292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:08.089292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:08.089292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:08.089292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:08.089292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:08.089292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:08.089292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:08.089292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:08.089292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:08.089292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:08.089292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:08.090186 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:28:08.090186 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:28:08.090186 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:28:08.090186 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:28:08.094425 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 519/1064] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:28:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:28:08.095470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4821663-0e7d-4609-adff-6b726f7f439a None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:08.197982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4821663-0e7d-4609-adff-6b726f7f439a None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:08.231450 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d043e2-8d6c-4cc0-9732-6b0f75dc210f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:08.232098 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d043e2-8d6c-4cc0-9732-6b0f75dc210f None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:08.232378 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d043e2-8d6c-4cc0-9732-6b0f75dc210f None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:08.232653 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d043e2-8d6c-4cc0-9732-6b0f75dc210f None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:08.233364 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d043e2-8d6c-4cc0-9732-6b0f75dc210f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:08.233667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d043e2-8d6c-4cc0-9732-6b0f75dc210f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:08.235055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d043e2-8d6c-4cc0-9732-6b0f75dc210f None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:08.251675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 546/1065] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:08.252936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff15fe-2360-4721-bdf4-c37449369356 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:08.288025 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a05bb-aa33-4cc7-9b3f-cb7b88b7c09c None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:08.288025 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a05bb-aa33-4cc7-9b3f-cb7b88b7c09c None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:08.288025 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a05bb-aa33-4cc7-9b3f-cb7b88b7c09c None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:08.288025 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 547/1066] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:28:08.290035 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ae210-401b-4a59-a2de-753cf3b295e7 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:08.290376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ae210-401b-4a59-a2de-753cf3b295e7 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:08.290624 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ae210-401b-4a59-a2de-753cf3b295e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:08.308846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-4b5ae210-401b-4a59-a2de-753cf3b295e7 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:08.310058 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-4b5ae210-401b-4a59-a2de-753cf3b295e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:08.315400 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:08.330840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:08.366244 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff15fe-2360-4721-bdf4-c37449369356 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:08.380475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 548/1067] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 18:28:08.401064 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf62c2e-8af3-4ad8-9a45-92c47b505ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:08.401845 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf62c2e-8af3-4ad8-9a45-92c47b505ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:08.402134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf62c2e-8af3-4ad8-9a45-92c47b505ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:08.402435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf62c2e-8af3-4ad8-9a45-92c47b505ee8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:08.402813 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ae210-401b-4a59-a2de-753cf3b295e7 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:08.411149 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf62c2e-8af3-4ad8-9a45-92c47b505ee8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:08.412721 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf62c2e-8af3-4ad8-9a45-92c47b505ee8 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:08.428193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-edf62c2e-8af3-4ad8-9a45-92c47b505ee8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 18:28:08.428193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:08.428193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:08.428193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:08.428193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:08.428193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:08.428193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:08.428193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:08.428193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:08.428193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:08.428193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:08.428193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:08.428193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:28:08.428193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:28:08.428193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:28:08.428193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:28:08.429196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:08.429196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:08.429196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:28:08.429196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:28:08.429196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:28:08.429196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:28:08.429196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:08.429196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:08.429196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:08.429196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:08.429196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:08.429196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:08.429196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:08.429196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:08.429196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:08.429196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:08.429196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:08.429196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:08.430759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:08.430759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:08.430759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:28:08.430759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:28:08.430759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:28:08.430759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:28:08.430759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:28:08.430759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:28:08.430759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 18:28:08.430759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:28:08.430759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 520/1068] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Jun 27 18:28:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:28:08.435674 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf62c2e-8af3-4ad8-9a45-92c47b505ee8 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:08.504250 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ae210-401b-4a59-a2de-753cf3b295e7 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:08.543208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be9f03-caa2-4c85-8988-7eac3b0831e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:08.543918 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be9f03-caa2-4c85-8988-7eac3b0831e1 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:08.544046 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf62c2e-8af3-4ad8-9a45-92c47b505ee8 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:08.544202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be9f03-caa2-4c85-8988-7eac3b0831e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:08.544478 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be9f03-caa2-4c85-8988-7eac3b0831e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:08.554732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be9f03-caa2-4c85-8988-7eac3b0831e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:28:08.560231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:08.562888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-a7be9f03-caa2-4c85-8988-7eac3b0831e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:28:08.562888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:08.562888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:08.562888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:08.562888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:08.562888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:08.562888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:08.562888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:08.562888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:08.562888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:08.562888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:08.562888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:08.562888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:28:08.562888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:28:08.562888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:28:08.562888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:28:08.564060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:28:08.564060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:28:08.564060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:28:08.564060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:28:08.564060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:08.564060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:08.564060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:08.564060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:08.564060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:08.564060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:08.564060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:08.564060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:08.564060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:08.564060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:08.564060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:08.564060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:08.565116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:08.565116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:08.565116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:08.565116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:08.565116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:28:08.565116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:28:08.565116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:28:08.565116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:28:08.565116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:28:08.565116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:28:08.565116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:28:08.565116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:28:08.565116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 549/1069] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:28:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:28:08.571370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be9f03-caa2-4c85-8988-7eac3b0831e1 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:08.588855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0dc83-9619-4d94-8104-f4ef0032a274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:08.589518 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0dc83-9619-4d94-8104-f4ef0032a274 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:08.589802 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0dc83-9619-4d94-8104-f4ef0032a274 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:08.590092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0dc83-9619-4d94-8104-f4ef0032a274 None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:08.590939 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0dc83-9619-4d94-8104-f4ef0032a274 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:08.591243 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0dc83-9619-4d94-8104-f4ef0032a274 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:08.592794 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0dc83-9619-4d94-8104-f4ef0032a274 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:08.593437 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:08.612248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 521/1070] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Jun 27 18:28:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:28:08.618227 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0dc83-9619-4d94-8104-f4ef0032a274 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:08.689567 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be9f03-caa2-4c85-8988-7eac3b0831e1 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:08.723749 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0dc83-9619-4d94-8104-f4ef0032a274 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:08.727984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90afc4-6451-4877-a5a4-fd2dca3fc452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:08.728641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90afc4-6451-4877-a5a4-fd2dca3fc452 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:08.728922 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90afc4-6451-4877-a5a4-fd2dca3fc452 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:08.729197 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90afc4-6451-4877-a5a4-fd2dca3fc452 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:08.729938 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90afc4-6451-4877-a5a4-fd2dca3fc452 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:08.730242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90afc4-6451-4877-a5a4-fd2dca3fc452 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:08.731911 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90afc4-6451-4877-a5a4-fd2dca3fc452 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:08.742112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 550/1071] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:28:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:28:08.749329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90afc4-6451-4877-a5a4-fd2dca3fc452 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:08.765620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69697986-abe4-4cc3-afe3-46217378acaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:08.766412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69697986-abe4-4cc3-afe3-46217378acaf None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:08.766716 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69697986-abe4-4cc3-afe3-46217378acaf None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:08.766989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69697986-abe4-4cc3-afe3-46217378acaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:08.777998 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69697986-abe4-4cc3-afe3-46217378acaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:08.779644 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69697986-abe4-4cc3-afe3-46217378acaf None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:08.797614 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-69697986-abe4-4cc3-afe3-46217378acaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:28:08.797614 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:08.797614 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:08.797614 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:08.797614 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:08.797614 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:08.797614 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:08.797614 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:08.797614 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:08.797614 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:08.797614 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:08.797614 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:08.797614 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:28:08.797614 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:28:08.797614 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:28:08.798569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:28:08.798569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:08.798569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:08.798569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:08.798569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:08.798569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:08.798569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:08.798569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:08.798569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:08.798569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:08.798569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:08.798569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:08.798569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:08.798569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:08.798569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:08.798569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:08.798569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:08.799502 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:28:08.799502 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:28:08.799502 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:28:08.799502 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:28:08.799502 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 522/1072] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:28:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:28:08.804291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69697986-abe4-4cc3-afe3-46217378acaf None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:08.830813 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:08.831376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:08.853395 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90afc4-6451-4877-a5a4-fd2dca3fc452 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:08.893866 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db5701-98a0-411a-aa9e-39a441f52437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:08.894759 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db5701-98a0-411a-aa9e-39a441f52437 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:08.895037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db5701-98a0-411a-aa9e-39a441f52437 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:08.895308 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db5701-98a0-411a-aa9e-39a441f52437 None admin] PATH_INFO: `/v3/users/2a7861ee7e4648dfa8be38b451b559f7` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:08.896626 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db5701-98a0-411a-aa9e-39a441f52437 None admin] RBAC: Authorizing `identity:get_user(user_id=2a7861ee7e4648dfa8be38b451b559f7)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:08.898055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db5701-98a0-411a-aa9e-39a441f52437 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:08.900536 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 551/1073] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:28:08 2022] GET /identity/v3/users/2a7861ee7e4648dfa8be38b451b559f7 => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:08.905929 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db5701-98a0-411a-aa9e-39a441f52437 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:08.911959 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69697986-abe4-4cc3-afe3-46217378acaf None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:08.949996 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd86580-5270-4f90-abaa-a7e090f1dc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:08.950684 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd86580-5270-4f90-abaa-a7e090f1dc6e None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:08.950967 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd86580-5270-4f90-abaa-a7e090f1dc6e None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:08.951240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd86580-5270-4f90-abaa-a7e090f1dc6e None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:08.951959 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd86580-5270-4f90-abaa-a7e090f1dc6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:08.952267 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd86580-5270-4f90-abaa-a7e090f1dc6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:08.953633 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd86580-5270-4f90-abaa-a7e090f1dc6e None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:08.971506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 523/1074] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:28:08.972604 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd86580-5270-4f90-abaa-a7e090f1dc6e None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:09.019590 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db5701-98a0-411a-aa9e-39a441f52437 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:09.057608 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98102c-6744-4f12-9886-a02678d699da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:09.058409 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98102c-6744-4f12-9886-a02678d699da None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:09.058707 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98102c-6744-4f12-9886-a02678d699da None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:09.058982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98102c-6744-4f12-9886-a02678d699da None admin] PATH_INFO: `/v3/projects/9fa2d394f79f4e5592f517d8a0f2b340` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:09.060225 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98102c-6744-4f12-9886-a02678d699da None admin] RBAC: Authorizing `identity:get_project(project_id=9fa2d394f79f4e5592f517d8a0f2b340)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:09.060863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:09.061661 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98102c-6744-4f12-9886-a02678d699da None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:09.063559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 552/1075] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:28:08 2022] GET /identity/v3/projects/9fa2d394f79f4e5592f517d8a0f2b340 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:09.069220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98102c-6744-4f12-9886-a02678d699da None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:09.077734 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd86580-5270-4f90-abaa-a7e090f1dc6e None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:09.085913 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:09.115042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee272cad-6db3-4310-8397-f603b686ab94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:09.115830 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee272cad-6db3-4310-8397-f603b686ab94 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:09.116113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee272cad-6db3-4310-8397-f603b686ab94 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:09.116398 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee272cad-6db3-4310-8397-f603b686ab94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:09.123329 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee272cad-6db3-4310-8397-f603b686ab94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:09.124768 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee272cad-6db3-4310-8397-f603b686ab94 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:09.140238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-ee272cad-6db3-4310-8397-f603b686ab94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:28:09.140238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:09.140238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:09.140238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:09.140238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:09.140238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:09.140238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:09.140238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:09.140238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:09.140238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:09.140238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:09.140238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:09.140238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:28:09.140238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:28:09.140238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:28:09.141337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:28:09.141337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:09.141337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:09.141337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:09.141337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:09.141337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:09.141337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:09.141337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:09.141337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:09.141337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:09.141337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:09.141337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:09.141337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:09.141337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:09.141337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:09.141337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:09.141337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:09.142683 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:28:09.142683 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:28:09.142683 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:28:09.142683 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:28:09.142683 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:28:09.142683 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:28:09.142683 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:28:09.142683 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:28:09.145867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 524/1076] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:28:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:28:09.146745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee272cad-6db3-4310-8397-f603b686ab94 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:09.183607 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98102c-6744-4f12-9886-a02678d699da None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:09.219545 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d6b75-f21f-4d68-8f18-0c00a70fb40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:09.220514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d6b75-f21f-4d68-8f18-0c00a70fb40f None admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:09.220698 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d6b75-f21f-4d68-8f18-0c00a70fb40f None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:09.220973 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d6b75-f21f-4d68-8f18-0c00a70fb40f None admin] PATH_INFO: `/v3/projects/9fa2d394f79f4e5592f517d8a0f2b340/users/2a7861ee7e4648dfa8be38b451b559f7/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:09.223032 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d6b75-f21f-4d68-8f18-0c00a70fb40f None admin] RBAC: Authorizing `identity:create_grant(project_id=9fa2d394f79f4e5592f517d8a0f2b340, user_id=2a7861ee7e4648dfa8be38b451b559f7, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:09.224457 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d6b75-f21f-4d68-8f18-0c00a70fb40f None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:09.253562 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee272cad-6db3-4310-8397-f603b686ab94 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:09.287874 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0399d-704c-49c3-b9af-c0e0a3b2e757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:09.288756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0399d-704c-49c3-b9af-c0e0a3b2e757 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:09.289045 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0399d-704c-49c3-b9af-c0e0a3b2e757 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:09.289322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0399d-704c-49c3-b9af-c0e0a3b2e757 None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:09.290668 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0399d-704c-49c3-b9af-c0e0a3b2e757 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:09.290958 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0399d-704c-49c3-b9af-c0e0a3b2e757 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:09.292383 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0399d-704c-49c3-b9af-c0e0a3b2e757 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:09.297313 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 553/1077] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:28:09 2022] PUT /identity/v3/projects/9fa2d394f79f4e5592f517d8a0f2b340/users/2a7861ee7e4648dfa8be38b451b559f7/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:28:09.316941 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 525/1078] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:28:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:09.317190 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d6b75-f21f-4d68-8f18-0c00a70fb40f None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:09.398325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:09.429354 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d6b75-f21f-4d68-8f18-0c00a70fb40f None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:09.466896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3d237-4eef-4554-96dd-bae986d6b4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:09.467865 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3d237-4eef-4554-96dd-bae986d6b4e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:09.468215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3d237-4eef-4554-96dd-bae986d6b4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:09.468550 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3d237-4eef-4554-96dd-bae986d6b4e7 None admin] PATH_INFO: `/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/1dddd4efb44c4152bb4151f7715f2223/roles/3ef93c53540740d6b89875c0ff507fab` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:09.470877 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3d237-4eef-4554-96dd-bae986d6b4e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=84ecf552f8364884b4aeff3428cb81e4, user_id=1dddd4efb44c4152bb4151f7715f2223, role_id=3ef93c53540740d6b89875c0ff507fab)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:09.472316 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3d237-4eef-4554-96dd-bae986d6b4e7 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:09.516978 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b3087-c3f9-4cf5-8533-7227c2ff7a7f None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:09.517259 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b3087-c3f9-4cf5-8533-7227c2ff7a7f None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:09.517536 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b3087-c3f9-4cf5-8533-7227c2ff7a7f None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:09.518947 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 526/1079] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:28:09.523982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0ee4c-354f-45e8-8429-c145efb0bde4 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:09.524264 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0ee4c-354f-45e8-8429-c145efb0bde4 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:09.524544 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0ee4c-354f-45e8-8429-c145efb0bde4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:09.530671 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:09.535913 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 554/1080] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:28:09 2022] PUT /identity/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/1dddd4efb44c4152bb4151f7715f2223/roles/3ef93c53540740d6b89875c0ff507fab => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:28:09.541354 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-16f0ee4c-354f-45e8-8429-c145efb0bde4 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:09.542607 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-16f0ee4c-354f-45e8-8429-c145efb0bde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:09.619985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 527/1081] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 18:28:09.642906 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3d237-4eef-4554-96dd-bae986d6b4e7 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:09.758545 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3d237-4eef-4554-96dd-bae986d6b4e7 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:09.793516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78251660-4ee8-4f31-bd05-673d4954000a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:09.794161 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78251660-4ee8-4f31-bd05-673d4954000a None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:09.794473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78251660-4ee8-4f31-bd05-673d4954000a None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:09.794752 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78251660-4ee8-4f31-bd05-673d4954000a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:09.797411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:09.863609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78251660-4ee8-4f31-bd05-673d4954000a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:09.865063 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78251660-4ee8-4f31-bd05-673d4954000a None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:09.962990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 555/1082] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:28:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 18:28:09.969629 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0ee4c-354f-45e8-8429-c145efb0bde4 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:10.069004 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0ee4c-354f-45e8-8429-c145efb0bde4 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:10.101771 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5d5d4-8aba-4bc4-ab54-5601a01a5e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:10.103083 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5d5d4-8aba-4bc4-ab54-5601a01a5e42 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:10.103361 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5d5d4-8aba-4bc4-ab54-5601a01a5e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:10.103633 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5d5d4-8aba-4bc4-ab54-5601a01a5e42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:10.105494 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:10.114441 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5d5d4-8aba-4bc4-ab54-5601a01a5e42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:10.115857 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5d5d4-8aba-4bc4-ab54-5601a01a5e42 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:10.132597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-99f5d5d4-8aba-4bc4-ab54-5601a01a5e42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:28:10.132597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:10.132597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:10.132597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:10.132597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:10.132597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:10.132597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:10.132597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:10.132597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:10.132597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:10.132597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:10.132597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:10.132597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:28:10.132597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:28:10.132597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:28:10.133953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:28:10.133953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:10.133953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:10.133953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:10.133953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:10.133953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:10.133953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:10.133953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:10.133953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:10.133953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:10.133953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:10.133953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:10.133953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:10.133953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:10.133953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:10.133953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:10.133953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:10.135204 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:28:10.135204 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:28:10.135204 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:28:10.135204 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:28:10.135822 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 528/1083] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:28:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:28:10.140568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78251660-4ee8-4f31-bd05-673d4954000a None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:10.174985 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:10.242206 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78251660-4ee8-4f31-bd05-673d4954000a None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:10.277551 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26087c31-67dd-436e-868f-24640c4d4f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:10.281743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26087c31-67dd-436e-868f-24640c4d4f5c None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:10.281743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26087c31-67dd-436e-868f-24640c4d4f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:10.281743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26087c31-67dd-436e-868f-24640c4d4f5c None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:10.281743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26087c31-67dd-436e-868f-24640c4d4f5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:10.281743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26087c31-67dd-436e-868f-24640c4d4f5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:10.281743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26087c31-67dd-436e-868f-24640c4d4f5c None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:10.302500 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 556/1084] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:28:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:11.354827 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3b185-ab24-4638-909a-ce8a01a16849 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:11.355364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3b185-ab24-4638-909a-ce8a01a16849 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:11.355364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3b185-ab24-4638-909a-ce8a01a16849 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:11.356995 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 529/1085] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:28:11.362347 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762dde0-3706-40fb-8604-cca169dd6891 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:11.362845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762dde0-3706-40fb-8604-cca169dd6891 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:11.362928 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762dde0-3706-40fb-8604-cca169dd6891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:11.379972 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-9762dde0-3706-40fb-8604-cca169dd6891 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:11.381146 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-9762dde0-3706-40fb-8604-cca169dd6891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:11.446842 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 557/1086] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:28:11.470215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3b185-ab24-4638-909a-ce8a01a16849 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:11.573961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3b185-ab24-4638-909a-ce8a01a16849 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:11.576174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9e234-f02c-473d-b321-5659cdf682b4 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:11.577244 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9e234-f02c-473d-b321-5659cdf682b4 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:11.577528 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9e234-f02c-473d-b321-5659cdf682b4 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:11.578942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 558/1087] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:28:11.584165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49215317-a6c0-4ffb-811d-078030699b5b None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:11.584447 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49215317-a6c0-4ffb-811d-078030699b5b None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:11.584719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49215317-a6c0-4ffb-811d-078030699b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:11.602500 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-49215317-a6c0-4ffb-811d-078030699b5b None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:11.603805 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-49215317-a6c0-4ffb-811d-078030699b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:11.609764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64821d-be8b-4d4a-a7f2-d52e0979b302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:11.610511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64821d-be8b-4d4a-a7f2-d52e0979b302 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:11.610795 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64821d-be8b-4d4a-a7f2-d52e0979b302 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:11.611070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64821d-be8b-4d4a-a7f2-d52e0979b302 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:11.622073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:11.622288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64821d-be8b-4d4a-a7f2-d52e0979b302 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:28:11.629841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-cc64821d-be8b-4d4a-a7f2-d52e0979b302 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:28:11.629841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:11.629841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:11.629841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:11.629841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:11.629841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:11.629841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:11.629841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:11.629841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:11.629841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:11.629841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:11.629841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:11.629841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:28:11.629841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:28:11.629841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:28:11.629841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:28:11.630807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:28:11.630807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:28:11.630807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:28:11.630807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:28:11.630807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:11.630807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:11.630807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:11.630807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:11.630807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:11.630807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:11.630807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:11.630807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:11.630807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:11.630807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:11.630807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:11.630807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:11.631800 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:11.631800 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:11.631800 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:11.631800 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:11.631800 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:28:11.631800 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:28:11.631800 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:28:11.631800 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:28:11.631800 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:28:11.631800 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:28:11.631800 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:28:11.631800 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:28:11.631800 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 530/1088] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:28:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:28:11.637074 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64821d-be8b-4d4a-a7f2-d52e0979b302 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:11.673338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 559/1089] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 18:28:11.684630 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:11.698569 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49215317-a6c0-4ffb-811d-078030699b5b None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:11.737449 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64821d-be8b-4d4a-a7f2-d52e0979b302 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:11.770479 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb98dd-c040-460c-aece-6bbc62c9d9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:11.771124 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb98dd-c040-460c-aece-6bbc62c9d9db None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:11.771404 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb98dd-c040-460c-aece-6bbc62c9d9db None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:11.771678 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb98dd-c040-460c-aece-6bbc62c9d9db None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:11.772419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb98dd-c040-460c-aece-6bbc62c9d9db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:11.772724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb98dd-c040-460c-aece-6bbc62c9d9db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:11.774770 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb98dd-c040-460c-aece-6bbc62c9d9db None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:11.783118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 531/1090] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:28:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:28:11.788736 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb98dd-c040-460c-aece-6bbc62c9d9db None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:11.800714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49215317-a6c0-4ffb-811d-078030699b5b None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:11.834078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4ab1e-93dc-41b9-8a38-8afad319217d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:11.834897 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4ab1e-93dc-41b9-8a38-8afad319217d None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:11.835177 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4ab1e-93dc-41b9-8a38-8afad319217d None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:11.835450 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4ab1e-93dc-41b9-8a38-8afad319217d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:11.845770 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4ab1e-93dc-41b9-8a38-8afad319217d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:28:11.854040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-ede4ab1e-93dc-41b9-8a38-8afad319217d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 18:28:11.854040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:11.854040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:11.854040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:11.854040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:11.854040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:11.854040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:11.854040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:11.854040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:11.854040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:11.854040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:11.854040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:11.854040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:28:11.854040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:28:11.854040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:28:11.854040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:28:11.855064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:28:11.855064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:28:11.855064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:28:11.855064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:28:11.855064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:11.855064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:11.855064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:11.855064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:11.855064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:11.855064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:11.855064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:11.855064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:11.855064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:11.855064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:11.855064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:11.855064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:11.856189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:11.856189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:11.856189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:11.856189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:11.856189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:28:11.856189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:28:11.856189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:28:11.856189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:28:11.856189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:28:11.856189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:28:11.856189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 18:28:11.856189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:28:11.856189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 560/1091] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Jun 27 18:28:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:28:11.861383 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4ab1e-93dc-41b9-8a38-8afad319217d None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:11.877179 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:11.890869 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb98dd-c040-460c-aece-6bbc62c9d9db None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:11.900405 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:11.926523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e26c6-5fd4-4170-a9c0-e196edd5f2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:11.927608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e26c6-5fd4-4170-a9c0-e196edd5f2ea None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:11.927887 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e26c6-5fd4-4170-a9c0-e196edd5f2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:11.928163 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e26c6-5fd4-4170-a9c0-e196edd5f2ea None admin] PATH_INFO: `/v3/users/2a7861ee7e4648dfa8be38b451b559f7` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:11.929361 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e26c6-5fd4-4170-a9c0-e196edd5f2ea None admin] RBAC: Authorizing `identity:get_user(user_id=2a7861ee7e4648dfa8be38b451b559f7)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:11.931445 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e26c6-5fd4-4170-a9c0-e196edd5f2ea None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:11.933103 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 532/1092] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:28:11 2022] GET /identity/v3/users/2a7861ee7e4648dfa8be38b451b559f7 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:11.938695 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e26c6-5fd4-4170-a9c0-e196edd5f2ea None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:11.965731 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4ab1e-93dc-41b9-8a38-8afad319217d None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:11.999266 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642cea9-69a6-41e3-ae52-ad104bb84eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:12.000119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642cea9-69a6-41e3-ae52-ad104bb84eec None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:12.000394 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642cea9-69a6-41e3-ae52-ad104bb84eec None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:12.000666 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642cea9-69a6-41e3-ae52-ad104bb84eec None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:12.001405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642cea9-69a6-41e3-ae52-ad104bb84eec None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:12.001706 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642cea9-69a6-41e3-ae52-ad104bb84eec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:12.003097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642cea9-69a6-41e3-ae52-ad104bb84eec None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:12.011663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 561/1093] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Jun 27 18:28:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:28:12.017208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642cea9-69a6-41e3-ae52-ad104bb84eec None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:12.037948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e26c6-5fd4-4170-a9c0-e196edd5f2ea None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:12.072121 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d583c66-09c5-4d90-82ec-73bd0e7c389f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:12.072746 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d583c66-09c5-4d90-82ec-73bd0e7c389f None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:12.073021 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d583c66-09c5-4d90-82ec-73bd0e7c389f None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:12.073297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d583c66-09c5-4d90-82ec-73bd0e7c389f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:12.092460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:12.114925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642cea9-69a6-41e3-ae52-ad104bb84eec None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:12.118019 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:12.139792 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d583c66-09c5-4d90-82ec-73bd0e7c389f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:12.141213 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d583c66-09c5-4d90-82ec-73bd0e7c389f None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:12.149346 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434cb8f5-a1f9-4732-8564-9512f0e417d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:12.150131 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cb8f5-a1f9-4732-8564-9512f0e417d3 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:12.150469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cb8f5-a1f9-4732-8564-9512f0e417d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:12.150745 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cb8f5-a1f9-4732-8564-9512f0e417d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:12.161023 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434cb8f5-a1f9-4732-8564-9512f0e417d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:12.162461 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434cb8f5-a1f9-4732-8564-9512f0e417d3 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:12.180692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-434cb8f5-a1f9-4732-8564-9512f0e417d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:28:12.180692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:12.180692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:12.180692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:12.180692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:12.180692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:12.180692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:12.180692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:12.180692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:12.180692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:12.180692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:12.180692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:12.180692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:28:12.180692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:28:12.180692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:28:12.181728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:28:12.181728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:12.181728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:12.181728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:12.181728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:12.181728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:12.181728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:12.181728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:12.181728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:12.181728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:12.181728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:12.181728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:12.181728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:12.181728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:12.181728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:12.181728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:12.181728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:12.182675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:28:12.182675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:28:12.182675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:28:12.182675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:28:12.182675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 562/1094] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:28:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:28:12.187507 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434cb8f5-a1f9-4732-8564-9512f0e417d3 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:12.241622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 533/1095] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:28:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 18:28:12.248209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d583c66-09c5-4d90-82ec-73bd0e7c389f None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:12.288035 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434cb8f5-a1f9-4732-8564-9512f0e417d3 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:12.314938 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:12.322630 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d56ba6-2f19-4aeb-963e-76c71848415b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:12.323282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d56ba6-2f19-4aeb-963e-76c71848415b None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:12.323566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d56ba6-2f19-4aeb-963e-76c71848415b None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:12.323841 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d56ba6-2f19-4aeb-963e-76c71848415b None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:12.324563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d56ba6-2f19-4aeb-963e-76c71848415b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:12.324952 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d56ba6-2f19-4aeb-963e-76c71848415b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:12.326329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d56ba6-2f19-4aeb-963e-76c71848415b None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:12.326747 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:12.337902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 563/1096] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:28:12.343508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d56ba6-2f19-4aeb-963e-76c71848415b None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:12.348845 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d583c66-09c5-4d90-82ec-73bd0e7c389f None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:12.383921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1aa3b-69a6-4ce3-86ba-b0c8311e7870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:12.384698 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1aa3b-69a6-4ce3-86ba-b0c8311e7870 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:12.384979 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1aa3b-69a6-4ce3-86ba-b0c8311e7870 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:12.385254 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1aa3b-69a6-4ce3-86ba-b0c8311e7870 None admin] PATH_INFO: `/v3/projects/9fa2d394f79f4e5592f517d8a0f2b340` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:12.386473 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b1aa3b-69a6-4ce3-86ba-b0c8311e7870 None admin] RBAC: Authorizing `identity:get_project(project_id=9fa2d394f79f4e5592f517d8a0f2b340)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:12.387880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b1aa3b-69a6-4ce3-86ba-b0c8311e7870 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:12.389565 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 534/1097] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:28:12 2022] GET /identity/v3/projects/9fa2d394f79f4e5592f517d8a0f2b340 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:28:12.395645 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1aa3b-69a6-4ce3-86ba-b0c8311e7870 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:12.445618 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d56ba6-2f19-4aeb-963e-76c71848415b None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:12.481064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb45bc-9050-4cc3-a472-22b6c6d0d0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:12.481838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb45bc-9050-4cc3-a472-22b6c6d0d0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:12.482120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb45bc-9050-4cc3-a472-22b6c6d0d0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:12.482424 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb45bc-9050-4cc3-a472-22b6c6d0d0d1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:12.489316 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eb45bc-9050-4cc3-a472-22b6c6d0d0d1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:12.490775 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eb45bc-9050-4cc3-a472-22b6c6d0d0d1 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:12.494458 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1aa3b-69a6-4ce3-86ba-b0c8311e7870 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:12.505713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-52eb45bc-9050-4cc3-a472-22b6c6d0d0d1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 18:28:12.505713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:12.505713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:12.505713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:12.505713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:12.505713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:12.505713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:12.505713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:12.505713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:12.505713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:12.505713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:12.505713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:12.505713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:28:12.505713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:28:12.505713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:28:12.505713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:28:12.506722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:12.506722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:12.506722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:28:12.506722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:28:12.506722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:28:12.506722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:28:12.506722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:12.506722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:12.506722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:12.506722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:12.506722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:12.506722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:12.506722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:12.506722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:12.506722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:12.506722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:12.506722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:12.506722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:12.507696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:12.507696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:12.507696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:28:12.507696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:28:12.507696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:28:12.507696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:28:12.507696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:28:12.507696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:28:12.507696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 18:28:12.507696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:28:12.507696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 564/1098] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Jun 27 18:28:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:28:12.512830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb45bc-9050-4cc3-a472-22b6c6d0d0d1 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:12.528879 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a92786-c985-4dce-aba5-9ae070da9d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:12.529523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a92786-c985-4dce-aba5-9ae070da9d2c None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:12.529804 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a92786-c985-4dce-aba5-9ae070da9d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:12.530105 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a92786-c985-4dce-aba5-9ae070da9d2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:12.530953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a92786-c985-4dce-aba5-9ae070da9d2c None admin] RBAC: Adding query filter params (role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f, scope.project.id=9fa2d394f79f4e5592f517d8a0f2b340, user.id=2a7861ee7e4648dfa8be38b451b559f7) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:12.531262 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a92786-c985-4dce-aba5-9ae070da9d2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:12.532679 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a92786-c985-4dce-aba5-9ae070da9d2c None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:12.540796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 535/1099] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:28:12 2022] GET /identity/v3/role_assignments?user.id=2a7861ee7e4648dfa8be38b451b559f7&scope.project.id=9fa2d394f79f4e5592f517d8a0f2b340&role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:28:12.577793 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:12.632399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb45bc-9050-4cc3-a472-22b6c6d0d0d1 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:12.667678 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b06bd6-9afc-408b-a4fe-b58eb5d935bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:12.668260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b06bd6-9afc-408b-a4fe-b58eb5d935bf None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:12.668545 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b06bd6-9afc-408b-a4fe-b58eb5d935bf None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:12.668821 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b06bd6-9afc-408b-a4fe-b58eb5d935bf None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:12.669665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b06bd6-9afc-408b-a4fe-b58eb5d935bf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:12.669975 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b06bd6-9afc-408b-a4fe-b58eb5d935bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:12.671457 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b06bd6-9afc-408b-a4fe-b58eb5d935bf None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:12.690274 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 565/1100] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Jun 27 18:28:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:28:12.697109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a92786-c985-4dce-aba5-9ae070da9d2c None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:12.710543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:12.799985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a92786-c985-4dce-aba5-9ae070da9d2c None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:12.834995 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8449b-a643-4649-b029-aeb80dd8fba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:12.835639 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8449b-a643-4649-b029-aeb80dd8fba0 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:12.835924 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8449b-a643-4649-b029-aeb80dd8fba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:12.836346 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8449b-a643-4649-b029-aeb80dd8fba0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:12.907981 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8449b-a643-4649-b029-aeb80dd8fba0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:12.909459 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8449b-a643-4649-b029-aeb80dd8fba0 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:12.919928 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:13.007174 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 536/1101] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:28:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 18:28:13.012865 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b06bd6-9afc-408b-a4fe-b58eb5d935bf None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:13.110819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b06bd6-9afc-408b-a4fe-b58eb5d935bf None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:13.118232 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:13.144211 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d139c-1987-41d5-b3de-79c5783c57ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:13.144990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d139c-1987-41d5-b3de-79c5783c57ad None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:13.145274 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d139c-1987-41d5-b3de-79c5783c57ad None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:13.145551 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d139c-1987-41d5-b3de-79c5783c57ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:13.155692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d139c-1987-41d5-b3de-79c5783c57ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:13.157234 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d139c-1987-41d5-b3de-79c5783c57ad None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:13.174700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-313d139c-1987-41d5-b3de-79c5783c57ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:28:13.174700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:13.174700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:13.174700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:13.174700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:13.174700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:13.174700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:13.174700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:13.174700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:13.174700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:13.174700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:13.174700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:13.174700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:28:13.174700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:28:13.174700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:28:13.175819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:28:13.175819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:13.175819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:13.175819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:13.175819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:13.175819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:13.175819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:13.175819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:13.175819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:13.175819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:13.175819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:13.175819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:13.175819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:13.175819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:13.175819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:13.175819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:13.175819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:13.177019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:28:13.177019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:28:13.177019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:28:13.177019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:28:13.178542 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 566/1102] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:28:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:28:13.181903 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8449b-a643-4649-b029-aeb80dd8fba0 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:13.280701 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8449b-a643-4649-b029-aeb80dd8fba0 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:13.289887 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:13.315056 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da9f89-622f-441b-9982-f8e731bba7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:13.315676 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da9f89-622f-441b-9982-f8e731bba7db None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:13.315960 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da9f89-622f-441b-9982-f8e731bba7db None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:13.316231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da9f89-622f-441b-9982-f8e731bba7db None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:13.316943 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da9f89-622f-441b-9982-f8e731bba7db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:13.317244 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da9f89-622f-441b-9982-f8e731bba7db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:13.318626 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da9f89-622f-441b-9982-f8e731bba7db None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:13.330607 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 537/1103] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:28:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:13.336154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d139c-1987-41d5-b3de-79c5783c57ad None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:13.447285 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d139c-1987-41d5-b3de-79c5783c57ad None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:13.480432 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00626a6-75d0-40ca-b54e-0c0601ddbd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:13.481447 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00626a6-75d0-40ca-b54e-0c0601ddbd86 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:13.481723 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00626a6-75d0-40ca-b54e-0c0601ddbd86 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:13.481994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00626a6-75d0-40ca-b54e-0c0601ddbd86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:13.488379 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00626a6-75d0-40ca-b54e-0c0601ddbd86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:13.489805 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00626a6-75d0-40ca-b54e-0c0601ddbd86 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:13.503255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-a00626a6-75d0-40ca-b54e-0c0601ddbd86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:28:13.503255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:13.503255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:13.503255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:13.503255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:13.503255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:13.503255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:13.503255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:13.503255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:13.503255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:13.503255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:13.503255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:13.503255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:28:13.503255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:28:13.503255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:28:13.504325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:28:13.504325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:13.504325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:13.504325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:13.504325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:13.504325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:13.504325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:13.504325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:13.504325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:13.504325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:13.504325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:13.504325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:13.504325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:13.504325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:13.504325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:13.504325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:13.504325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:13.505260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:28:13.505260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:28:13.505260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:28:13.505260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:28:13.505260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:28:13.505260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:28:13.505260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:28:13.505260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:28:13.505260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 567/1104] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:28:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:28:13.510068 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da9f89-622f-441b-9982-f8e731bba7db None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:13.608494 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da9f89-622f-441b-9982-f8e731bba7db None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:13.641158 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedc33e-31c6-4fcb-bc28-68e0029ca70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:13.641798 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedc33e-31c6-4fcb-bc28-68e0029ca70b None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:13.642108 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedc33e-31c6-4fcb-bc28-68e0029ca70b None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:13.642338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedc33e-31c6-4fcb-bc28-68e0029ca70b None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:13.643136 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eedc33e-31c6-4fcb-bc28-68e0029ca70b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:13.643438 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eedc33e-31c6-4fcb-bc28-68e0029ca70b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:13.644845 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eedc33e-31c6-4fcb-bc28-68e0029ca70b None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:13.657162 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 538/1105] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:28:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:13.663092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00626a6-75d0-40ca-b54e-0c0601ddbd86 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:13.683643 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:13.767933 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00626a6-75d0-40ca-b54e-0c0601ddbd86 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:13.801505 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a827eab-b3b4-4caf-87a0-b6b66c3f7d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:13.802141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a827eab-b3b4-4caf-87a0-b6b66c3f7d72 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:13.802445 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a827eab-b3b4-4caf-87a0-b6b66c3f7d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:13.802725 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a827eab-b3b4-4caf-87a0-b6b66c3f7d72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:13.803539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a827eab-b3b4-4caf-87a0-b6b66c3f7d72 None admin] RBAC: Adding query filter params (role.id=3ef93c53540740d6b89875c0ff507fab, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=1dddd4efb44c4152bb4151f7715f2223) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:13.803847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a827eab-b3b4-4caf-87a0-b6b66c3f7d72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:13.805752 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a827eab-b3b4-4caf-87a0-b6b66c3f7d72 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:13.811398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 568/1106] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:28:13 2022] GET /identity/v3/role_assignments?user.id=1dddd4efb44c4152bb4151f7715f2223&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=3ef93c53540740d6b89875c0ff507fab => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:14.620089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d1615-d9d6-45cf-be48-24712c6aa747 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:14.620594 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d1615-d9d6-45cf-be48-24712c6aa747 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:14.620703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d1615-d9d6-45cf-be48-24712c6aa747 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:14.622209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 539/1107] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:28:14.627842 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34966d-8a38-49ac-8056-b68b46f5454c None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:14.628272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34966d-8a38-49ac-8056-b68b46f5454c None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:14.628401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34966d-8a38-49ac-8056-b68b46f5454c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:14.647291 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-dc34966d-8a38-49ac-8056-b68b46f5454c None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:14.648513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-dc34966d-8a38-49ac-8056-b68b46f5454c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:14.714432 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 569/1108] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:28:14.739654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d1615-d9d6-45cf-be48-24712c6aa747 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:14.767251 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:14.838792 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d1615-d9d6-45cf-be48-24712c6aa747 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:14.871545 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e602c363-c805-4e7c-a45b-24898e07d7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:14.872255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602c363-c805-4e7c-a45b-24898e07d7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:14.872530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602c363-c805-4e7c-a45b-24898e07d7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:14.872801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602c363-c805-4e7c-a45b-24898e07d7b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:14.873948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e602c363-c805-4e7c-a45b-24898e07d7b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:14.875380 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e602c363-c805-4e7c-a45b-24898e07d7b0 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:14.876970 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 540/1109] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:28:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:14.882703 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34966d-8a38-49ac-8056-b68b46f5454c None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:14.896541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:14.979449 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34966d-8a38-49ac-8056-b68b46f5454c None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:15.012375 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd0a7b-aa8f-44be-8f43-b24f2886b793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:15.013019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd0a7b-aa8f-44be-8f43-b24f2886b793 None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:15.013296 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd0a7b-aa8f-44be-8f43-b24f2886b793 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:15.013567 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd0a7b-aa8f-44be-8f43-b24f2886b793 None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:15.014252 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd0a7b-aa8f-44be-8f43-b24f2886b793 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:15.015695 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd0a7b-aa8f-44be-8f43-b24f2886b793 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:15.093034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 570/1110] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Jun 27 18:28:14 2022] POST /identity/v3/users => generated 286 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:28:17.151225 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e88108-e50f-45e5-a43d-63cf330b55be None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:17.151776 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e88108-e50f-45e5-a43d-63cf330b55be None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:17.151776 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e88108-e50f-45e5-a43d-63cf330b55be None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:17.153129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 541/1111] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:28:17.158224 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83da11d-1f27-41ee-a9fb-c4b455bbbabc None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:17.158677 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83da11d-1f27-41ee-a9fb-c4b455bbbabc None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:17.158807 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83da11d-1f27-41ee-a9fb-c4b455bbbabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:17.175976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-a83da11d-1f27-41ee-a9fb-c4b455bbbabc None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:17.177150 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-a83da11d-1f27-41ee-a9fb-c4b455bbbabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:17.236755 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:17.241974 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 571/1112] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:28:17.266094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e88108-e50f-45e5-a43d-63cf330b55be None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:17.359690 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:17.366138 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e88108-e50f-45e5-a43d-63cf330b55be None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:17.399464 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bf3d5-d31e-47f2-8c25-2fe85b0fa6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:17.400298 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bf3d5-d31e-47f2-8c25-2fe85b0fa6fb None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:17.400580 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bf3d5-d31e-47f2-8c25-2fe85b0fa6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:17.400857 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bf3d5-d31e-47f2-8c25-2fe85b0fa6fb None admin] PATH_INFO: `/v3/roles/44d7d6bfe7eb441bbc6a301a5b6784ad` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:17.402371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1bf3d5-d31e-47f2-8c25-2fe85b0fa6fb None admin] RBAC: Authorizing `identity:get_role(role_id=44d7d6bfe7eb441bbc6a301a5b6784ad)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:17.403735 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1bf3d5-d31e-47f2-8c25-2fe85b0fa6fb None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:17.405095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 542/1113] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:28:17 2022] GET /identity/v3/roles/44d7d6bfe7eb441bbc6a301a5b6784ad => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:17.410904 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83da11d-1f27-41ee-a9fb-c4b455bbbabc None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:17.516469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83da11d-1f27-41ee-a9fb-c4b455bbbabc None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:17.549886 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e9aa6-637c-4446-83b0-40d633c7f426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:17.550837 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e9aa6-637c-4446-83b0-40d633c7f426 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:17.551115 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e9aa6-637c-4446-83b0-40d633c7f426 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:17.551384 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e9aa6-637c-4446-83b0-40d633c7f426 None admin] PATH_INFO: `/v3/users/b824942f001347a4bcbe723e1b17626c` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:17.571900 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e9aa6-637c-4446-83b0-40d633c7f426 None admin] RBAC: Authorizing `identity:get_user(user_id=b824942f001347a4bcbe723e1b17626c)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:17.573414 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7e9aa6-637c-4446-83b0-40d633c7f426 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:17.575103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 572/1114] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:28:17 2022] GET /identity/v3/users/b824942f001347a4bcbe723e1b17626c => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:17.580927 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bf3d5-d31e-47f2-8c25-2fe85b0fa6fb None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:17.679265 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bf3d5-d31e-47f2-8c25-2fe85b0fa6fb None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:17.712386 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770599d7-28d7-4c74-a7f5-bd9feec7d097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:17.713012 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770599d7-28d7-4c74-a7f5-bd9feec7d097 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:17.713292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770599d7-28d7-4c74-a7f5-bd9feec7d097 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:17.713569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770599d7-28d7-4c74-a7f5-bd9feec7d097 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:17.743631 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:17.780956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770599d7-28d7-4c74-a7f5-bd9feec7d097 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:17.782376 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770599d7-28d7-4c74-a7f5-bd9feec7d097 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:17.879810 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 543/1115] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:28:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 18:28:17.886821 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e9aa6-637c-4446-83b0-40d633c7f426 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:17.940134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:17.981451 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7e9aa6-637c-4446-83b0-40d633c7f426 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:18.014177 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b81beb-b680-4865-9de0-f3eb9e56b972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:18.015041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b81beb-b680-4865-9de0-f3eb9e56b972 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:18.015320 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b81beb-b680-4865-9de0-f3eb9e56b972 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:18.015588 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b81beb-b680-4865-9de0-f3eb9e56b972 None admin] PATH_INFO: `/v3/projects/9fa2d394f79f4e5592f517d8a0f2b340` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:18.016737 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b81beb-b680-4865-9de0-f3eb9e56b972 None admin] RBAC: Authorizing `identity:get_project(project_id=9fa2d394f79f4e5592f517d8a0f2b340)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:18.018150 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b81beb-b680-4865-9de0-f3eb9e56b972 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:18.019835 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 573/1116] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:28:17 2022] GET /identity/v3/projects/9fa2d394f79f4e5592f517d8a0f2b340 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:18.026576 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770599d7-28d7-4c74-a7f5-bd9feec7d097 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:18.086879 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:18.122532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770599d7-28d7-4c74-a7f5-bd9feec7d097 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:18.155791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9517313-d2f9-4197-ad7c-f953f0c4fe1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:18.156423 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9517313-d2f9-4197-ad7c-f953f0c4fe1e None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:18.156708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9517313-d2f9-4197-ad7c-f953f0c4fe1e None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:18.156984 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9517313-d2f9-4197-ad7c-f953f0c4fe1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:18.157811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9517313-d2f9-4197-ad7c-f953f0c4fe1e None admin] RBAC: Adding query filter params (role.id=44d7d6bfe7eb441bbc6a301a5b6784ad, scope.project.id=9fa2d394f79f4e5592f517d8a0f2b340, user.id=b824942f001347a4bcbe723e1b17626c) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:18.158125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9517313-d2f9-4197-ad7c-f953f0c4fe1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:18.159580 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9517313-d2f9-4197-ad7c-f953f0c4fe1e None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:18.165443 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 544/1117] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:28:18 2022] GET /identity/v3/role_assignments?user.id=b824942f001347a4bcbe723e1b17626c&scope.project.id=9fa2d394f79f4e5592f517d8a0f2b340&role.id=44d7d6bfe7eb441bbc6a301a5b6784ad => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:20.180561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8e611-e599-41d2-b4fe-79ed7cccfd2f None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:20.181136 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8e611-e599-41d2-b4fe-79ed7cccfd2f None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:20.181136 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8e611-e599-41d2-b4fe-79ed7cccfd2f None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:20.182465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 574/1118] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:28:20.187668 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa03b1a-4f45-46f9-b05d-371a914e7314 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:20.188045 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa03b1a-4f45-46f9-b05d-371a914e7314 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:20.188224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa03b1a-4f45-46f9-b05d-371a914e7314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:20.204471 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-5fa03b1a-4f45-46f9-b05d-371a914e7314 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:20.205644 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-5fa03b1a-4f45-46f9-b05d-371a914e7314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:20.270707 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 545/1119] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:28:20.295223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8e611-e599-41d2-b4fe-79ed7cccfd2f None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:20.391784 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8e611-e599-41d2-b4fe-79ed7cccfd2f None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:20.414923 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:20.424801 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0451f-de66-4b26-ac2e-53b4b158e1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:20.425632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0451f-de66-4b26-ac2e-53b4b158e1af None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:20.425906 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0451f-de66-4b26-ac2e-53b4b158e1af None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:20.426176 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0451f-de66-4b26-ac2e-53b4b158e1af None admin] PATH_INFO: `/v3/roles/44d7d6bfe7eb441bbc6a301a5b6784ad` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:20.427650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0451f-de66-4b26-ac2e-53b4b158e1af None admin] RBAC: Authorizing `identity:get_role(role_id=44d7d6bfe7eb441bbc6a301a5b6784ad)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:20.434767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0451f-de66-4b26-ac2e-53b4b158e1af None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:20.434767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 575/1120] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:28:20 2022] GET /identity/v3/roles/44d7d6bfe7eb441bbc6a301a5b6784ad => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:20.436111 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa03b1a-4f45-46f9-b05d-371a914e7314 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:20.493081 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:20.533834 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa03b1a-4f45-46f9-b05d-371a914e7314 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:20.565447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58d532-0c31-486b-866c-af5cbd5b2903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:20.566289 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58d532-0c31-486b-866c-af5cbd5b2903 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:20.566626 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58d532-0c31-486b-866c-af5cbd5b2903 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:20.566901 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58d532-0c31-486b-866c-af5cbd5b2903 None admin] PATH_INFO: `/v3/users/b824942f001347a4bcbe723e1b17626c` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:20.568054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58d532-0c31-486b-866c-af5cbd5b2903 None admin] RBAC: Authorizing `identity:get_user(user_id=b824942f001347a4bcbe723e1b17626c)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:20.569493 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58d532-0c31-486b-866c-af5cbd5b2903 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:20.571791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 546/1121] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:28:20 2022] GET /identity/v3/users/b824942f001347a4bcbe723e1b17626c => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:20.577872 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0451f-de66-4b26-ac2e-53b4b158e1af None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:20.674812 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0451f-de66-4b26-ac2e-53b4b158e1af None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:20.707727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db0fcb-8452-4cdf-ae58-0cb68ce50f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:20.708494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db0fcb-8452-4cdf-ae58-0cb68ce50f55 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:20.708771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db0fcb-8452-4cdf-ae58-0cb68ce50f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:20.709040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db0fcb-8452-4cdf-ae58-0cb68ce50f55 None admin] PATH_INFO: `/v3/projects/9fa2d394f79f4e5592f517d8a0f2b340` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:20.710190 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db0fcb-8452-4cdf-ae58-0cb68ce50f55 None admin] RBAC: Authorizing `identity:get_project(project_id=9fa2d394f79f4e5592f517d8a0f2b340)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:20.711649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db0fcb-8452-4cdf-ae58-0cb68ce50f55 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:20.713302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 576/1122] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:28:20 2022] GET /identity/v3/projects/9fa2d394f79f4e5592f517d8a0f2b340 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:20.719134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58d532-0c31-486b-866c-af5cbd5b2903 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:20.818094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58d532-0c31-486b-866c-af5cbd5b2903 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:20.844179 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:20.852637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9530a1-60a0-4af4-aebf-ebcb1f4835aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:20.858530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9530a1-60a0-4af4-aebf-ebcb1f4835aa None admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:20.858530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9530a1-60a0-4af4-aebf-ebcb1f4835aa None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:20.858530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9530a1-60a0-4af4-aebf-ebcb1f4835aa None admin] PATH_INFO: `/v3/projects/9fa2d394f79f4e5592f517d8a0f2b340/users/b824942f001347a4bcbe723e1b17626c/roles/44d7d6bfe7eb441bbc6a301a5b6784ad` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:20.858530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9530a1-60a0-4af4-aebf-ebcb1f4835aa None admin] RBAC: Authorizing `identity:create_grant(project_id=9fa2d394f79f4e5592f517d8a0f2b340, user_id=b824942f001347a4bcbe723e1b17626c, role_id=44d7d6bfe7eb441bbc6a301a5b6784ad)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:20.858530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9530a1-60a0-4af4-aebf-ebcb1f4835aa None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:20.969150 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 547/1123] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:28:20 2022] PUT /identity/v3/projects/9fa2d394f79f4e5592f517d8a0f2b340/users/b824942f001347a4bcbe723e1b17626c/roles/44d7d6bfe7eb441bbc6a301a5b6784ad => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:28:22.958200 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba29a3-0604-4efe-aabf-b867da2a5ae7 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:22.958755 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba29a3-0604-4efe-aabf-b867da2a5ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:22.958842 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba29a3-0604-4efe-aabf-b867da2a5ae7 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:22.960151 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 577/1124] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:28:22.965810 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175a423-0d2a-493d-b2f7-176a22197ff8 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:22.966246 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175a423-0d2a-493d-b2f7-176a22197ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:22.966353 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175a423-0d2a-493d-b2f7-176a22197ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:22.983631 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-c175a423-0d2a-493d-b2f7-176a22197ff8 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:22.984807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-c175a423-0d2a-493d-b2f7-176a22197ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:23.048446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 548/1125] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:28:23.071441 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba29a3-0604-4efe-aabf-b867da2a5ae7 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:23.130142 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:23.170753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba29a3-0604-4efe-aabf-b867da2a5ae7 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:23.202920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b2187-f0e9-44d6-aa7d-93be4741211d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:23.203750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b2187-f0e9-44d6-aa7d-93be4741211d None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:23.204036 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b2187-f0e9-44d6-aa7d-93be4741211d None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:23.204312 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b2187-f0e9-44d6-aa7d-93be4741211d None admin] PATH_INFO: `/v3/roles/44d7d6bfe7eb441bbc6a301a5b6784ad` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:23.205770 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b2187-f0e9-44d6-aa7d-93be4741211d None admin] RBAC: Authorizing `identity:get_role(role_id=44d7d6bfe7eb441bbc6a301a5b6784ad)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:23.207163 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b2187-f0e9-44d6-aa7d-93be4741211d None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:23.208521 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 578/1126] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:28:23 2022] GET /identity/v3/roles/44d7d6bfe7eb441bbc6a301a5b6784ad => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:23.214799 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175a423-0d2a-493d-b2f7-176a22197ff8 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:23.310758 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175a423-0d2a-493d-b2f7-176a22197ff8 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:23.326976 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:23.344234 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90472bf-f142-4d5b-a7ac-792df0213324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:23.345123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90472bf-f142-4d5b-a7ac-792df0213324 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:23.345406 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90472bf-f142-4d5b-a7ac-792df0213324 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:23.345679 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90472bf-f142-4d5b-a7ac-792df0213324 None admin] PATH_INFO: `/v3/users/b824942f001347a4bcbe723e1b17626c` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:23.346887 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90472bf-f142-4d5b-a7ac-792df0213324 None admin] RBAC: Authorizing `identity:get_user(user_id=b824942f001347a4bcbe723e1b17626c)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:23.348306 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90472bf-f142-4d5b-a7ac-792df0213324 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:23.349979 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 549/1127] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:28:23 2022] GET /identity/v3/users/b824942f001347a4bcbe723e1b17626c => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:23.356532 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b2187-f0e9-44d6-aa7d-93be4741211d None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:23.453313 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b2187-f0e9-44d6-aa7d-93be4741211d None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:23.479035 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:23.486751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a006ea5-55e0-47b3-bd6d-96d352b80d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:23.487383 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a006ea5-55e0-47b3-bd6d-96d352b80d93 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:23.487669 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a006ea5-55e0-47b3-bd6d-96d352b80d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:23.497326 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a006ea5-55e0-47b3-bd6d-96d352b80d93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:23.552954 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a006ea5-55e0-47b3-bd6d-96d352b80d93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:23.555247 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a006ea5-55e0-47b3-bd6d-96d352b80d93 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:23.649921 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 579/1128] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:28:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 18:28:23.655665 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90472bf-f142-4d5b-a7ac-792df0213324 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:23.752597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90472bf-f142-4d5b-a7ac-792df0213324 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:23.785531 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df2aa9-294e-4156-bb96-44526d8009ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:23.786362 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df2aa9-294e-4156-bb96-44526d8009ca None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:23.786675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df2aa9-294e-4156-bb96-44526d8009ca None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:23.786950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df2aa9-294e-4156-bb96-44526d8009ca None admin] PATH_INFO: `/v3/projects/9fa2d394f79f4e5592f517d8a0f2b340` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:23.788109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df2aa9-294e-4156-bb96-44526d8009ca None admin] RBAC: Authorizing `identity:get_project(project_id=9fa2d394f79f4e5592f517d8a0f2b340)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:23.790159 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df2aa9-294e-4156-bb96-44526d8009ca None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:23.791879 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 550/1129] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:28:23 2022] GET /identity/v3/projects/9fa2d394f79f4e5592f517d8a0f2b340 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:23.797859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a006ea5-55e0-47b3-bd6d-96d352b80d93 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:23.824498 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:23.893175 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a006ea5-55e0-47b3-bd6d-96d352b80d93 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:23.926711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43130d00-b28a-4b30-9e94-49465dc2e401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:23.927288 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43130d00-b28a-4b30-9e94-49465dc2e401 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:23.927572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43130d00-b28a-4b30-9e94-49465dc2e401 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:23.927849 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43130d00-b28a-4b30-9e94-49465dc2e401 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:23.928677 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43130d00-b28a-4b30-9e94-49465dc2e401 None admin] RBAC: Adding query filter params (role.id=44d7d6bfe7eb441bbc6a301a5b6784ad, scope.project.id=9fa2d394f79f4e5592f517d8a0f2b340, user.id=b824942f001347a4bcbe723e1b17626c) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:23.928989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43130d00-b28a-4b30-9e94-49465dc2e401 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:23.930982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43130d00-b28a-4b30-9e94-49465dc2e401 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:23.936663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 580/1130] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:28:23 2022] GET /identity/v3/role_assignments?user.id=b824942f001347a4bcbe723e1b17626c&scope.project.id=9fa2d394f79f4e5592f517d8a0f2b340&role.id=44d7d6bfe7eb441bbc6a301a5b6784ad => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:25.964031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d69d3e-1202-4ea3-885f-75747a7694a0 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:25.964579 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d69d3e-1202-4ea3-885f-75747a7694a0 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:25.964579 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d69d3e-1202-4ea3-885f-75747a7694a0 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:25.965930 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 551/1131] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:28:25.971110 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab89b6-4017-4555-aca7-8a063b53e9cb None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:25.971508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab89b6-4017-4555-aca7-8a063b53e9cb None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:25.971665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab89b6-4017-4555-aca7-8a063b53e9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:25.989020 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-ccab89b6-4017-4555-aca7-8a063b53e9cb None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:25.990197 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ccab89b6-4017-4555-aca7-8a063b53e9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:26.055526 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 581/1132] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:28:26.079249 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d69d3e-1202-4ea3-885f-75747a7694a0 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:26.131275 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:26.182251 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d69d3e-1202-4ea3-885f-75747a7694a0 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:26.216080 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dbb3e2-be7d-4a51-a59a-9ded1a419e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:26.216856 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbb3e2-be7d-4a51-a59a-9ded1a419e2c None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:26.217134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbb3e2-be7d-4a51-a59a-9ded1a419e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:26.217406 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbb3e2-be7d-4a51-a59a-9ded1a419e2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:26.218584 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dbb3e2-be7d-4a51-a59a-9ded1a419e2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:26.220007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dbb3e2-be7d-4a51-a59a-9ded1a419e2c None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:26.221611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 552/1133] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:28:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:26.227706 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab89b6-4017-4555-aca7-8a063b53e9cb None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:26.252471 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:26.328105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab89b6-4017-4555-aca7-8a063b53e9cb None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:26.362208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afde4b-be0b-4a6e-8f12-be1c8e6b6c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:26.362889 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afde4b-be0b-4a6e-8f12-be1c8e6b6c4b None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:26.363172 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afde4b-be0b-4a6e-8f12-be1c8e6b6c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:26.363448 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afde4b-be0b-4a6e-8f12-be1c8e6b6c4b None admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:26.364162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afde4b-be0b-4a6e-8f12-be1c8e6b6c4b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:26.365591 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afde4b-be0b-4a6e-8f12-be1c8e6b6c4b None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:26.497288 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 582/1134] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Jun 27 18:28:26 2022] POST /identity/v3/projects => generated 311 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:28:29.144453 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d51da-2696-4824-945a-57c7c480dd77 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:29.145009 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d51da-2696-4824-945a-57c7c480dd77 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:29.145009 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d51da-2696-4824-945a-57c7c480dd77 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:29.146477 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 553/1135] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:28:29.151859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f315e-f17b-4521-85ad-1271f3d4e3eb None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:29.152299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f315e-f17b-4521-85ad-1271f3d4e3eb None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:29.152420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f315e-f17b-4521-85ad-1271f3d4e3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:29.168713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-f97f315e-f17b-4521-85ad-1271f3d4e3eb None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:29.169914 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-f97f315e-f17b-4521-85ad-1271f3d4e3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:29.234082 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 583/1136] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 18:28:29.259383 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d51da-2696-4824-945a-57c7c480dd77 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:29.359566 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d51da-2696-4824-945a-57c7c480dd77 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:29.379942 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:29.393496 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f633082-e311-47a8-bf64-ebd1175b1c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:29.394272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f633082-e311-47a8-bf64-ebd1175b1c47 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:29.394633 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f633082-e311-47a8-bf64-ebd1175b1c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:29.394859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f633082-e311-47a8-bf64-ebd1175b1c47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:29.396052 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f633082-e311-47a8-bf64-ebd1175b1c47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:29.397464 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f633082-e311-47a8-bf64-ebd1175b1c47 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:29.399093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 554/1137] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:28:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:29.404834 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f315e-f17b-4521-85ad-1271f3d4e3eb None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:29.407975 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:29.511126 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f315e-f17b-4521-85ad-1271f3d4e3eb None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:29.543432 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2eeeb4-3b45-4452-89c7-4b920d2b6bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:29.544084 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2eeeb4-3b45-4452-89c7-4b920d2b6bfc None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:29.544368 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2eeeb4-3b45-4452-89c7-4b920d2b6bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:29.544644 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2eeeb4-3b45-4452-89c7-4b920d2b6bfc None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:29.545339 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2eeeb4-3b45-4452-89c7-4b920d2b6bfc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:29.546804 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2eeeb4-3b45-4452-89c7-4b920d2b6bfc None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:29.613560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 584/1138] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Jun 27 18:28:29 2022] POST /identity/v3/users => generated 286 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:28:31.653191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a53e88-c278-4594-96f6-97b86f14105e None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:31.653751 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a53e88-c278-4594-96f6-97b86f14105e None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:31.653751 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a53e88-c278-4594-96f6-97b86f14105e None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:31.655127 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 555/1139] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:28:31.660164 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9d5ed-5dbd-435f-97a0-eee1eb7ccabe None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:31.660599 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9d5ed-5dbd-435f-97a0-eee1eb7ccabe None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:31.660692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9d5ed-5dbd-435f-97a0-eee1eb7ccabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:31.677765 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-90c9d5ed-5dbd-435f-97a0-eee1eb7ccabe None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:31.678967 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-90c9d5ed-5dbd-435f-97a0-eee1eb7ccabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:31.734132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:31.746015 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 585/1140] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:28:31.769425 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a53e88-c278-4594-96f6-97b86f14105e None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:31.867343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a53e88-c278-4594-96f6-97b86f14105e None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:31.901544 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6cde1-9045-4253-b418-83ef17e7bfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:31.902311 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6cde1-9045-4253-b418-83ef17e7bfc2 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:31.902639 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6cde1-9045-4253-b418-83ef17e7bfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:31.902922 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6cde1-9045-4253-b418-83ef17e7bfc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:31.912752 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6cde1-9045-4253-b418-83ef17e7bfc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:28:31.920796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-4dc6cde1-9045-4253-b418-83ef17e7bfc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:28:31.920796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:31.920796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:31.920796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:31.920796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:31.920796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:31.920796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:31.920796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:31.920796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:31.920796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:31.920796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:31.920796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:31.920796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:28:31.920796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:28:31.920796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:28:31.920796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:28:31.921948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:28:31.921948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:28:31.921948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:28:31.921948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:28:31.921948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:31.921948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:31.921948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:31.921948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:31.921948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:31.921948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:31.921948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:31.921948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:31.921948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:31.921948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:31.921948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:31.921948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:31.922875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:31.922875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:31.922875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:31.922875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:31.922875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:28:31.922875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:28:31.922875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:28:31.922875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:28:31.922875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:28:31.922875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:28:31.922875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:28:31.922875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:28:31.922875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 556/1141] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:28:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:28:31.928698 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c9d5ed-5dbd-435f-97a0-eee1eb7ccabe None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:32.022777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c9d5ed-5dbd-435f-97a0-eee1eb7ccabe None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:32.055662 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79047971-c01d-43b2-8c92-d1292b435367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:32.056300 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79047971-c01d-43b2-8c92-d1292b435367 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:32.056580 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79047971-c01d-43b2-8c92-d1292b435367 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:32.056853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79047971-c01d-43b2-8c92-d1292b435367 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:32.057586 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79047971-c01d-43b2-8c92-d1292b435367 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:32.057888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79047971-c01d-43b2-8c92-d1292b435367 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:32.059268 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79047971-c01d-43b2-8c92-d1292b435367 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:32.066946 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 586/1142] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:28:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:32.072795 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6cde1-9045-4253-b418-83ef17e7bfc2 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:32.121382 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:32.170612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6cde1-9045-4253-b418-83ef17e7bfc2 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:32.201841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307fa32-cec6-4a15-8aa5-a4b00dd88548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:32.202708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307fa32-cec6-4a15-8aa5-a4b00dd88548 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:32.202991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307fa32-cec6-4a15-8aa5-a4b00dd88548 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:32.203267 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307fa32-cec6-4a15-8aa5-a4b00dd88548 None admin] PATH_INFO: `/v3/users/9a997fed9c1643e7a615003729f39250` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:32.216349 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307fa32-cec6-4a15-8aa5-a4b00dd88548 None admin] RBAC: Authorizing `identity:get_user(user_id=9a997fed9c1643e7a615003729f39250)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:32.217773 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307fa32-cec6-4a15-8aa5-a4b00dd88548 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:32.219422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 557/1143] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:28:32 2022] GET /identity/v3/users/9a997fed9c1643e7a615003729f39250 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:32.225089 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79047971-c01d-43b2-8c92-d1292b435367 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:32.279090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:32.324447 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79047971-c01d-43b2-8c92-d1292b435367 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:32.367296 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5a1f0-18ba-47a2-afdf-67b9f09959cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:32.368121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5a1f0-18ba-47a2-afdf-67b9f09959cd None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:32.368406 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5a1f0-18ba-47a2-afdf-67b9f09959cd None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:32.368682 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5a1f0-18ba-47a2-afdf-67b9f09959cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:32.433301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5a1f0-18ba-47a2-afdf-67b9f09959cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:32.434688 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5a1f0-18ba-47a2-afdf-67b9f09959cd None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:32.484028 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:32.530496 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 587/1144] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:28:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 18:28:32.536496 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307fa32-cec6-4a15-8aa5-a4b00dd88548 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:32.632914 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307fa32-cec6-4a15-8aa5-a4b00dd88548 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:32.640460 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:32.666526 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c953e1-520f-418b-af8c-d0306d1ce05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:32.667363 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c953e1-520f-418b-af8c-d0306d1ce05d None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:32.667642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c953e1-520f-418b-af8c-d0306d1ce05d None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:32.667921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c953e1-520f-418b-af8c-d0306d1ce05d None admin] PATH_INFO: `/v3/projects/50ca1f9c2f2b42ce93a1e028d9d273f4` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:32.669114 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c953e1-520f-418b-af8c-d0306d1ce05d None admin] RBAC: Authorizing `identity:get_project(project_id=50ca1f9c2f2b42ce93a1e028d9d273f4)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:32.670565 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c953e1-520f-418b-af8c-d0306d1ce05d None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:32.672247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 558/1145] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:28:32 2022] GET /identity/v3/projects/50ca1f9c2f2b42ce93a1e028d9d273f4 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:32.678313 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5a1f0-18ba-47a2-afdf-67b9f09959cd None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:32.776175 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5a1f0-18ba-47a2-afdf-67b9f09959cd None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:32.808828 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9cf55c-e2c6-4177-9b17-2e0e4c9ac234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:32.809460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9cf55c-e2c6-4177-9b17-2e0e4c9ac234 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:32.809747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9cf55c-e2c6-4177-9b17-2e0e4c9ac234 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:32.810024 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9cf55c-e2c6-4177-9b17-2e0e4c9ac234 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:32.810878 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9cf55c-e2c6-4177-9b17-2e0e4c9ac234 None admin] RBAC: Adding query filter params (role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f, scope.project.id=50ca1f9c2f2b42ce93a1e028d9d273f4, user.id=9a997fed9c1643e7a615003729f39250) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:32.811191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9cf55c-e2c6-4177-9b17-2e0e4c9ac234 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:32.812617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9cf55c-e2c6-4177-9b17-2e0e4c9ac234 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:32.817869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 588/1146] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:28:32 2022] GET /identity/v3/role_assignments?user.id=9a997fed9c1643e7a615003729f39250&scope.project.id=50ca1f9c2f2b42ce93a1e028d9d273f4&role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:34.824491 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82934806-d7ad-456d-8878-90e213af8850 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:34.825003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82934806-d7ad-456d-8878-90e213af8850 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:34.825109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82934806-d7ad-456d-8878-90e213af8850 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:34.826432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 559/1147] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:28:34.831682 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629723e3-ed48-4869-92a3-57f74060d8f0 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:34.832110 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629723e3-ed48-4869-92a3-57f74060d8f0 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:34.832240 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629723e3-ed48-4869-92a3-57f74060d8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:34.854531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-629723e3-ed48-4869-92a3-57f74060d8f0 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:34.855997 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-629723e3-ed48-4869-92a3-57f74060d8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:34.867073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:34.922176 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 589/1148] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:28:34.945420 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82934806-d7ad-456d-8878-90e213af8850 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:35.055687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82934806-d7ad-456d-8878-90e213af8850 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:35.090432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a10648-96ec-42db-ac73-1219c9e55608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:35.091412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a10648-96ec-42db-ac73-1219c9e55608 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:35.091931 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a10648-96ec-42db-ac73-1219c9e55608 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:35.092447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a10648-96ec-42db-ac73-1219c9e55608 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:35.106819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a10648-96ec-42db-ac73-1219c9e55608 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:28:35.114371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-58a10648-96ec-42db-ac73-1219c9e55608 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:28:35.114371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:35.114371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:35.114371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:35.114371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:35.114371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:35.114371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:35.114371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:35.114371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:35.114371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:35.114371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:35.114371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:35.114371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:28:35.114371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:28:35.114371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:28:35.114371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:28:35.115511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:28:35.115511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:28:35.115511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:28:35.115511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:28:35.115511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:35.115511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:35.115511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:35.115511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:35.115511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:35.115511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:35.115511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:35.115511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:35.115511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:35.115511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:35.115511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:35.115511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:35.116704 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:35.116704 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:35.116704 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:35.116704 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:35.116704 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:28:35.116704 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:28:35.116704 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:28:35.116704 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:28:35.116704 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:28:35.116704 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:28:35.116704 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:28:35.116704 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:28:35.116704 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 560/1149] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:28:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:28:35.122008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629723e3-ed48-4869-92a3-57f74060d8f0 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:35.228875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629723e3-ed48-4869-92a3-57f74060d8f0 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:35.265501 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79410e0-9873-4ead-833c-981ac2da8073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:35.266165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79410e0-9873-4ead-833c-981ac2da8073 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:35.266462 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79410e0-9873-4ead-833c-981ac2da8073 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:35.266732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79410e0-9873-4ead-833c-981ac2da8073 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:35.267477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79410e0-9873-4ead-833c-981ac2da8073 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:35.267780 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79410e0-9873-4ead-833c-981ac2da8073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:35.269136 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79410e0-9873-4ead-833c-981ac2da8073 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:35.279029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 590/1150] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:28:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:35.284976 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a10648-96ec-42db-ac73-1219c9e55608 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:35.322780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:35.396094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a10648-96ec-42db-ac73-1219c9e55608 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:35.433168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06f032-f794-4c7b-acbc-709539a37fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:35.434581 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06f032-f794-4c7b-acbc-709539a37fac None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:35.434864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06f032-f794-4c7b-acbc-709539a37fac None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:35.435136 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06f032-f794-4c7b-acbc-709539a37fac None admin] PATH_INFO: `/v3/users/9a997fed9c1643e7a615003729f39250` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:35.436357 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06f032-f794-4c7b-acbc-709539a37fac None admin] RBAC: Authorizing `identity:get_user(user_id=9a997fed9c1643e7a615003729f39250)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:35.437769 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06f032-f794-4c7b-acbc-709539a37fac None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:35.439627 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 561/1151] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:28:35 2022] GET /identity/v3/users/9a997fed9c1643e7a615003729f39250 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:35.446321 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79410e0-9873-4ead-833c-981ac2da8073 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:35.458581 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:35.560891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79410e0-9873-4ead-833c-981ac2da8073 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:35.597272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cb96e-24d6-4ab1-b51d-1d78d409315b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:35.598452 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cb96e-24d6-4ab1-b51d-1d78d409315b None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:35.598999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cb96e-24d6-4ab1-b51d-1d78d409315b None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:35.599288 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cb96e-24d6-4ab1-b51d-1d78d409315b None admin] PATH_INFO: `/v3/projects/50ca1f9c2f2b42ce93a1e028d9d273f4` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:35.600504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081cb96e-24d6-4ab1-b51d-1d78d409315b None admin] RBAC: Authorizing `identity:get_project(project_id=50ca1f9c2f2b42ce93a1e028d9d273f4)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:35.601973 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081cb96e-24d6-4ab1-b51d-1d78d409315b None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:35.603762 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 591/1152] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:28:35 2022] GET /identity/v3/projects/50ca1f9c2f2b42ce93a1e028d9d273f4 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:35.610045 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06f032-f794-4c7b-acbc-709539a37fac None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:35.709467 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06f032-f794-4c7b-acbc-709539a37fac None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:35.711005 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:35.744142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640c5cc-1167-4b96-a4d5-a3f50b12bae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:35.745026 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640c5cc-1167-4b96-a4d5-a3f50b12bae2 None admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:35.745303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640c5cc-1167-4b96-a4d5-a3f50b12bae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:35.745574 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640c5cc-1167-4b96-a4d5-a3f50b12bae2 None admin] PATH_INFO: `/v3/projects/50ca1f9c2f2b42ce93a1e028d9d273f4/users/9a997fed9c1643e7a615003729f39250/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:35.747570 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640c5cc-1167-4b96-a4d5-a3f50b12bae2 None admin] RBAC: Authorizing `identity:create_grant(project_id=50ca1f9c2f2b42ce93a1e028d9d273f4, user_id=9a997fed9c1643e7a615003729f39250, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:35.748996 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640c5cc-1167-4b96-a4d5-a3f50b12bae2 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:35.819572 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 562/1153] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:28:35 2022] PUT /identity/v3/projects/50ca1f9c2f2b42ce93a1e028d9d273f4/users/9a997fed9c1643e7a615003729f39250/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:28:37.813019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c84a97-9f01-47bd-8eba-e71e0341c5a5 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:37.813517 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c84a97-9f01-47bd-8eba-e71e0341c5a5 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:37.813627 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c84a97-9f01-47bd-8eba-e71e0341c5a5 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:37.814958 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 592/1154] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:28:37.820090 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689374f8-57a3-44a8-b8bf-81898b07cf8a None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:37.820580 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689374f8-57a3-44a8-b8bf-81898b07cf8a None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:37.820864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689374f8-57a3-44a8-b8bf-81898b07cf8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:37.836759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-689374f8-57a3-44a8-b8bf-81898b07cf8a None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:37.837919 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-689374f8-57a3-44a8-b8bf-81898b07cf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:37.902533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 563/1155] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 18:28:37.927655 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c84a97-9f01-47bd-8eba-e71e0341c5a5 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:38.002334 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:38.024765 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c84a97-9f01-47bd-8eba-e71e0341c5a5 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:38.057165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7d450-f65d-478f-8f81-904840b59cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:38.057925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7d450-f65d-478f-8f81-904840b59cae None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:38.058205 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7d450-f65d-478f-8f81-904840b59cae None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:38.058504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7d450-f65d-478f-8f81-904840b59cae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:38.068186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7d450-f65d-478f-8f81-904840b59cae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:28:38.075939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-93f7d450-f65d-478f-8f81-904840b59cae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:28:38.075939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:38.075939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:38.075939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:38.075939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:38.075939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:38.075939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:38.075939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:38.075939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:38.075939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:38.075939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:38.075939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:38.075939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:28:38.075939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:28:38.075939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:28:38.075939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:28:38.076887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:28:38.076887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:28:38.076887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:28:38.076887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:28:38.076887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:38.076887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:38.076887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:38.076887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:38.076887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:38.076887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:38.076887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:38.076887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:38.076887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:38.076887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:38.076887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:38.076887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:38.077942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:38.077942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:38.077942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:38.077942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:38.077942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:28:38.077942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:28:38.077942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:28:38.077942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:28:38.077942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:28:38.077942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:28:38.077942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:28:38.077942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:28:38.077942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 593/1156] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:28:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:28:38.082958 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689374f8-57a3-44a8-b8bf-81898b07cf8a None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:38.176052 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:38.184719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689374f8-57a3-44a8-b8bf-81898b07cf8a None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:38.217710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cafd8-7015-42de-a35e-a100ff22e4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:38.218402 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cafd8-7015-42de-a35e-a100ff22e4fb None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:38.218694 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cafd8-7015-42de-a35e-a100ff22e4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:38.218962 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cafd8-7015-42de-a35e-a100ff22e4fb None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:38.219689 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cafd8-7015-42de-a35e-a100ff22e4fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:38.219988 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cafd8-7015-42de-a35e-a100ff22e4fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:38.221324 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cafd8-7015-42de-a35e-a100ff22e4fb None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:38.229761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 564/1157] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:28:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:28:38.235528 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7d450-f65d-478f-8f81-904840b59cae None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:38.332863 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7d450-f65d-478f-8f81-904840b59cae None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:38.366559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4fdc22-22a6-4648-89e0-555773f5d897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:38.367408 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fdc22-22a6-4648-89e0-555773f5d897 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:38.367687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fdc22-22a6-4648-89e0-555773f5d897 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:38.367960 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fdc22-22a6-4648-89e0-555773f5d897 None admin] PATH_INFO: `/v3/users/9a997fed9c1643e7a615003729f39250` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:38.369138 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4fdc22-22a6-4648-89e0-555773f5d897 None admin] RBAC: Authorizing `identity:get_user(user_id=9a997fed9c1643e7a615003729f39250)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:38.370604 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4fdc22-22a6-4648-89e0-555773f5d897 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:38.372272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 594/1158] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:28:38 2022] GET /identity/v3/users/9a997fed9c1643e7a615003729f39250 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:38.377919 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cafd8-7015-42de-a35e-a100ff22e4fb None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:38.474490 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cafd8-7015-42de-a35e-a100ff22e4fb None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:38.506455 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f489b-c44e-4c26-8798-95ef08c2204e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:38.507079 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f489b-c44e-4c26-8798-95ef08c2204e None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:38.507360 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f489b-c44e-4c26-8798-95ef08c2204e None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:38.508160 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f489b-c44e-4c26-8798-95ef08c2204e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:38.531354 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:38.574644 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f489b-c44e-4c26-8798-95ef08c2204e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:38.576065 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f489b-c44e-4c26-8798-95ef08c2204e None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:38.669329 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 565/1159] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:28:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 18:28:38.675121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4fdc22-22a6-4648-89e0-555773f5d897 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:38.678805 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:38.773878 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4fdc22-22a6-4648-89e0-555773f5d897 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:38.806158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16af22-011a-4829-9e77-7672a90f39d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:38.807033 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16af22-011a-4829-9e77-7672a90f39d5 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:38.807319 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16af22-011a-4829-9e77-7672a90f39d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:38.807592 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16af22-011a-4829-9e77-7672a90f39d5 None admin] PATH_INFO: `/v3/projects/50ca1f9c2f2b42ce93a1e028d9d273f4` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:38.808762 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16af22-011a-4829-9e77-7672a90f39d5 None admin] RBAC: Authorizing `identity:get_project(project_id=50ca1f9c2f2b42ce93a1e028d9d273f4)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:38.810195 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16af22-011a-4829-9e77-7672a90f39d5 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:38.811919 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 595/1160] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:28:38 2022] GET /identity/v3/projects/50ca1f9c2f2b42ce93a1e028d9d273f4 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:38.817968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f489b-c44e-4c26-8798-95ef08c2204e None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:38.871912 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:38.913811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f489b-c44e-4c26-8798-95ef08c2204e None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:38.947001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f9837-bbe9-48b1-abe9-954642a37ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:38.947573 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f9837-bbe9-48b1-abe9-954642a37ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:38.947854 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f9837-bbe9-48b1-abe9-954642a37ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:38.948129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f9837-bbe9-48b1-abe9-954642a37ab5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:38.948942 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f9837-bbe9-48b1-abe9-954642a37ab5 None admin] RBAC: Adding query filter params (role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f, scope.project.id=50ca1f9c2f2b42ce93a1e028d9d273f4, user.id=9a997fed9c1643e7a615003729f39250) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:38.949251 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f9837-bbe9-48b1-abe9-954642a37ab5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:38.950681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f9837-bbe9-48b1-abe9-954642a37ab5 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:38.956326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 566/1161] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:28:38 2022] GET /identity/v3/role_assignments?user.id=9a997fed9c1643e7a615003729f39250&scope.project.id=50ca1f9c2f2b42ce93a1e028d9d273f4&role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:40.977628 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb6375-3a81-4973-a352-e8bf850506b0 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:40.978175 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb6375-3a81-4973-a352-e8bf850506b0 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:40.978175 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb6375-3a81-4973-a352-e8bf850506b0 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:40.979582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 596/1162] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:28:40.985116 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ef179-9c9a-4cbb-9df5-eabfe4481d89 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:40.985535 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ef179-9c9a-4cbb-9df5-eabfe4481d89 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:40.985671 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ef179-9c9a-4cbb-9df5-eabfe4481d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:41.003487 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-fe9ef179-9c9a-4cbb-9df5-eabfe4481d89 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:41.004596 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-fe9ef179-9c9a-4cbb-9df5-eabfe4481d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:41.069170 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 567/1163] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:28:41.092331 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb6375-3a81-4973-a352-e8bf850506b0 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:41.161676 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:41.191071 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb6375-3a81-4973-a352-e8bf850506b0 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:41.224732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5355f-bfae-4d88-9452-2ff78f93c238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:41.225353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5355f-bfae-4d88-9452-2ff78f93c238 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:41.225633 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5355f-bfae-4d88-9452-2ff78f93c238 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:41.225904 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5355f-bfae-4d88-9452-2ff78f93c238 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:41.292035 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5355f-bfae-4d88-9452-2ff78f93c238 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:41.293392 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5355f-bfae-4d88-9452-2ff78f93c238 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:41.358221 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:41.387736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 597/1164] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:28:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 18:28:41.393740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ef179-9c9a-4cbb-9df5-eabfe4481d89 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:41.440711 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:41.489231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ef179-9c9a-4cbb-9df5-eabfe4481d89 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:41.522327 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67772a-d43c-4feb-8683-ad2c8296c7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:41.523152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67772a-d43c-4feb-8683-ad2c8296c7cc None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:41.523433 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67772a-d43c-4feb-8683-ad2c8296c7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:41.523707 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67772a-d43c-4feb-8683-ad2c8296c7cc None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:41.530170 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67772a-d43c-4feb-8683-ad2c8296c7cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:41.531630 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67772a-d43c-4feb-8683-ad2c8296c7cc None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:41.546001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-2e67772a-d43c-4feb-8683-ad2c8296c7cc None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 27 18:28:41.546001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:41.546001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 27 18:28:41.546001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 27 18:28:41.546001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:28:41.546001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:28:41.546001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:28:41.546001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:28:41.546001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 27 18:28:41.546001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:28:41.546001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:28:41.546001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:28:41.546001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:41.546001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:41.546001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:41.546001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:41.546001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:41.547280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:41.547280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:41.547280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:41.547280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:41.547280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:41.547280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:41.547280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:28:41.547280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:28:41.547280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:28:41.547280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:28:41.547280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:41.547280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:41.547280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:41.547280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:41.547280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:41.547280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:41.548421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:41.548421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:41.548421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:41.548421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:41.548421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:41.548421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:41.548421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:41.548421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:41.548421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:41.548421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:41.548421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 27 18:28:41.548421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:28:41.548421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 27 18:28:41.548421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:28:41.548421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 568/1165] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Jun 27 18:28:41 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:28:41.553185 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5355f-bfae-4d88-9452-2ff78f93c238 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:41.652093 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5355f-bfae-4d88-9452-2ff78f93c238 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:41.685178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15024356-d433-4cc1-85a3-0ffaeb49d143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:41.685812 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15024356-d433-4cc1-85a3-0ffaeb49d143 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:41.686091 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15024356-d433-4cc1-85a3-0ffaeb49d143 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:41.686362 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15024356-d433-4cc1-85a3-0ffaeb49d143 None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:41.687095 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15024356-d433-4cc1-85a3-0ffaeb49d143 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:41.687394 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15024356-d433-4cc1-85a3-0ffaeb49d143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:41.688743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15024356-d433-4cc1-85a3-0ffaeb49d143 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:41.696144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 598/1166] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Jun 27 18:28:41 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:43.688685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be6535-805a-4fbd-a51d-2f539aef871c None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:43.689246 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be6535-805a-4fbd-a51d-2f539aef871c None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:43.689246 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be6535-805a-4fbd-a51d-2f539aef871c None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:43.690626 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 569/1167] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:28:43.695760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d5188-20fa-467f-bd73-af18dfdb5a5e None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:43.696206 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d5188-20fa-467f-bd73-af18dfdb5a5e None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:43.696308 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d5188-20fa-467f-bd73-af18dfdb5a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:43.712782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-396d5188-20fa-467f-bd73-af18dfdb5a5e None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:43.713954 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-396d5188-20fa-467f-bd73-af18dfdb5a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:43.742924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:43.779665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 599/1168] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:28:43.803326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be6535-805a-4fbd-a51d-2f539aef871c None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:43.902492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be6535-805a-4fbd-a51d-2f539aef871c None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:43.911148 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:43.935047 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aabd465-a543-4e5b-bddc-ced1bcc9c438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:43.935698 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabd465-a543-4e5b-bddc-ced1bcc9c438 None admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:43.935980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabd465-a543-4e5b-bddc-ced1bcc9c438 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:43.936251 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabd465-a543-4e5b-bddc-ced1bcc9c438 None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:43.937023 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aabd465-a543-4e5b-bddc-ced1bcc9c438 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:43.938416 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aabd465-a543-4e5b-bddc-ced1bcc9c438 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:44.002646 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 570/1169] 10.222.0.15 () {60 vars in 1138 bytes} [Mon Jun 27 18:28:43 2022] POST /identity/v3/domains => generated 267 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:28:46.035968 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aaf066-4b88-4e21-a52f-6f7b51f96aad None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:46.036474 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aaf066-4b88-4e21-a52f-6f7b51f96aad None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:46.036474 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aaf066-4b88-4e21-a52f-6f7b51f96aad None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:46.037814 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 600/1170] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:28:46.043646 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b46c00-0e9b-44ad-a6f8-bc8139616fc3 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:46.044038 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b46c00-0e9b-44ad-a6f8-bc8139616fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:46.044302 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b46c00-0e9b-44ad-a6f8-bc8139616fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:46.061162 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-08b46c00-0e9b-44ad-a6f8-bc8139616fc3 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:46.062327 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-08b46c00-0e9b-44ad-a6f8-bc8139616fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:46.126669 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 571/1171] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:28:46.150490 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aaf066-4b88-4e21-a52f-6f7b51f96aad None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:46.247792 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aaf066-4b88-4e21-a52f-6f7b51f96aad None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:46.280666 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1716976-202f-45bd-aab4-3f21f634dfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:46.286578 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1716976-202f-45bd-aab4-3f21f634dfef None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:46.286578 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1716976-202f-45bd-aab4-3f21f634dfef None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:46.286578 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1716976-202f-45bd-aab4-3f21f634dfef None admin] PATH_INFO: `/v3/domains/f5aafb9d681a49e7984fb16fff3fc2f6` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:46.292310 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1716976-202f-45bd-aab4-3f21f634dfef None admin] RBAC: Authorizing `identity:get_domain(domain_id=f5aafb9d681a49e7984fb16fff3fc2f6)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:46.293742 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1716976-202f-45bd-aab4-3f21f634dfef None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:46.295457 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 601/1172] 10.222.0.15 () {56 vars in 1183 bytes} [Mon Jun 27 18:28:46 2022] GET /identity/v3/domains/f5aafb9d681a49e7984fb16fff3fc2f6 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:46.301356 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b46c00-0e9b-44ad-a6f8-bc8139616fc3 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:46.394981 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:46.399056 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b46c00-0e9b-44ad-a6f8-bc8139616fc3 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:46.432272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e39a0d-ee69-4c43-b8ff-b42fbc816a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:46.432904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e39a0d-ee69-4c43-b8ff-b42fbc816a78 None admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:46.433180 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e39a0d-ee69-4c43-b8ff-b42fbc816a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:46.433451 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e39a0d-ee69-4c43-b8ff-b42fbc816a78 None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:46.434142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e39a0d-ee69-4c43-b8ff-b42fbc816a78 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:46.435635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e39a0d-ee69-4c43-b8ff-b42fbc816a78 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:46.511898 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 572/1173] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Jun 27 18:28:46 2022] POST /identity/v3/projects => generated 361 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:28:48.564523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5756fa1-8de1-409f-b697-2f60139fde62 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:48.565085 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5756fa1-8de1-409f-b697-2f60139fde62 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:48.565085 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5756fa1-8de1-409f-b697-2f60139fde62 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:48.566510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 602/1174] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:28:48.572593 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3d5ea-6f86-4ada-aa56-7808dd70080f None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:48.573017 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3d5ea-6f86-4ada-aa56-7808dd70080f None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:48.573143 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3d5ea-6f86-4ada-aa56-7808dd70080f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:48.589384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-e8b3d5ea-6f86-4ada-aa56-7808dd70080f None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:48.590595 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-e8b3d5ea-6f86-4ada-aa56-7808dd70080f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:48.655967 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 573/1175] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:28:48.680374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5756fa1-8de1-409f-b697-2f60139fde62 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:48.708690 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:48.780624 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5756fa1-8de1-409f-b697-2f60139fde62 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:48.813430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd306e-02bc-4335-ab8f-77291ad8481b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:48.814269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd306e-02bc-4335-ab8f-77291ad8481b None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:48.814626 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd306e-02bc-4335-ab8f-77291ad8481b None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:48.814840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd306e-02bc-4335-ab8f-77291ad8481b None admin] PATH_INFO: `/v3/domains/f5aafb9d681a49e7984fb16fff3fc2f6` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:48.815997 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd306e-02bc-4335-ab8f-77291ad8481b None admin] RBAC: Authorizing `identity:get_domain(domain_id=f5aafb9d681a49e7984fb16fff3fc2f6)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:48.817411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd306e-02bc-4335-ab8f-77291ad8481b None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:48.819028 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 603/1176] 10.222.0.15 () {56 vars in 1183 bytes} [Mon Jun 27 18:28:48 2022] GET /identity/v3/domains/f5aafb9d681a49e7984fb16fff3fc2f6 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:48.825107 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3d5ea-6f86-4ada-aa56-7808dd70080f None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:48.903652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:48.922275 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3d5ea-6f86-4ada-aa56-7808dd70080f None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:48.955468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346a7d9-1a12-4ada-9030-7ab07bc9f7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:48.956112 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346a7d9-1a12-4ada-9030-7ab07bc9f7ee None admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:48.956391 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346a7d9-1a12-4ada-9030-7ab07bc9f7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:48.956665 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346a7d9-1a12-4ada-9030-7ab07bc9f7ee None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:48.957353 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9346a7d9-1a12-4ada-9030-7ab07bc9f7ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:48.958782 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9346a7d9-1a12-4ada-9030-7ab07bc9f7ee None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:49.064408 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 574/1177] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Jun 27 18:28:48 2022] POST /identity/v3/users => generated 311 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:28:51.108663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5bce8-e586-4fee-8c07-cc4279819849 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:51.109292 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5bce8-e586-4fee-8c07-cc4279819849 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:51.109292 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5bce8-e586-4fee-8c07-cc4279819849 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:51.110692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 604/1178] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:28:51.115853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145be9a2-307e-41a8-b180-a76460211ad3 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:51.116236 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145be9a2-307e-41a8-b180-a76460211ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:51.116409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145be9a2-307e-41a8-b180-a76460211ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:51.133336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-145be9a2-307e-41a8-b180-a76460211ad3 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:51.134714 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-145be9a2-307e-41a8-b180-a76460211ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:51.199431 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 575/1179] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:28:51.222259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5bce8-e586-4fee-8c07-cc4279819849 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:51.313535 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:51.320716 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5bce8-e586-4fee-8c07-cc4279819849 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:51.353190 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c30356-60ec-49fe-81f2-7c5b2c5227c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:51.353947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c30356-60ec-49fe-81f2-7c5b2c5227c9 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:51.354225 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c30356-60ec-49fe-81f2-7c5b2c5227c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:51.354528 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c30356-60ec-49fe-81f2-7c5b2c5227c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:51.364881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c30356-60ec-49fe-81f2-7c5b2c5227c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:28:51.372998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-82c30356-60ec-49fe-81f2-7c5b2c5227c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:28:51.372998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:51.372998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:51.372998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:51.372998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:51.372998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:51.372998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:51.372998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:51.372998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:51.372998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:51.372998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:51.372998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:51.372998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:28:51.372998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:28:51.372998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:28:51.372998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:28:51.374133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:28:51.374133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:28:51.374133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:28:51.374133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:28:51.374133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:51.374133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:51.374133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:51.374133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:51.374133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:51.374133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:51.374133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:51.374133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:51.374133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:51.374133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:51.374133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:51.374133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:51.375338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:51.375338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:51.375338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:51.375338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:51.375338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:28:51.375338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:28:51.375338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:28:51.375338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:28:51.375338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:28:51.375338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:28:51.375338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:28:51.375338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:28:51.375338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 605/1180] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:28:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:28:51.380138 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145be9a2-307e-41a8-b180-a76460211ad3 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:51.443917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:51.478727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145be9a2-307e-41a8-b180-a76460211ad3 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:51.512006 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f03c05-dd09-4f78-892a-1559b9139796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:51.512643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f03c05-dd09-4f78-892a-1559b9139796 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:51.512923 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f03c05-dd09-4f78-892a-1559b9139796 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:51.513196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f03c05-dd09-4f78-892a-1559b9139796 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:51.513924 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f03c05-dd09-4f78-892a-1559b9139796 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:51.514226 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f03c05-dd09-4f78-892a-1559b9139796 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:51.515597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f03c05-dd09-4f78-892a-1559b9139796 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:51.523328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 576/1181] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:28:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:51.529824 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c30356-60ec-49fe-81f2-7c5b2c5227c9 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:51.626575 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c30356-60ec-49fe-81f2-7c5b2c5227c9 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:51.658963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc60851-5de4-44e1-acf7-7b58aec9fedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:51.659808 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc60851-5de4-44e1-acf7-7b58aec9fedb None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:51.660089 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc60851-5de4-44e1-acf7-7b58aec9fedb None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:51.660360 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc60851-5de4-44e1-acf7-7b58aec9fedb None admin] PATH_INFO: `/v3/users/92ad46c03cf94f4682fe29e3c3684917` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:51.673648 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc60851-5de4-44e1-acf7-7b58aec9fedb None admin] RBAC: Authorizing `identity:get_user(user_id=92ad46c03cf94f4682fe29e3c3684917)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:51.675116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc60851-5de4-44e1-acf7-7b58aec9fedb None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:51.676748 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 606/1182] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:28:51 2022] GET /identity/v3/users/92ad46c03cf94f4682fe29e3c3684917 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:51.682605 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f03c05-dd09-4f78-892a-1559b9139796 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:51.781174 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f03c05-dd09-4f78-892a-1559b9139796 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:51.805863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:51.813298 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c130b8-7852-42e5-bc07-a53bdb632f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:51.815749 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c130b8-7852-42e5-bc07-a53bdb632f45 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:51.815749 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c130b8-7852-42e5-bc07-a53bdb632f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:51.815749 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c130b8-7852-42e5-bc07-a53bdb632f45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:51.879659 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c130b8-7852-42e5-bc07-a53bdb632f45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:51.881029 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c130b8-7852-42e5-bc07-a53bdb632f45 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:51.975017 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 577/1183] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:28:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 18:28:51.981548 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc60851-5de4-44e1-acf7-7b58aec9fedb None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:51.996571 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:52.080346 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc60851-5de4-44e1-acf7-7b58aec9fedb None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:52.112855 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24191858-0c68-446d-9242-72a03ff0a936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:52.113693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24191858-0c68-446d-9242-72a03ff0a936 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:52.113973 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24191858-0c68-446d-9242-72a03ff0a936 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:52.114246 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24191858-0c68-446d-9242-72a03ff0a936 None admin] PATH_INFO: `/v3/projects/396f54b3e5f54e83a0ed520cb04cb77b` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:52.115429 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24191858-0c68-446d-9242-72a03ff0a936 None admin] RBAC: Authorizing `identity:get_project(project_id=396f54b3e5f54e83a0ed520cb04cb77b)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:52.116852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24191858-0c68-446d-9242-72a03ff0a936 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:52.119213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 607/1184] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:28:51 2022] GET /identity/v3/projects/396f54b3e5f54e83a0ed520cb04cb77b => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:52.125241 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c130b8-7852-42e5-bc07-a53bdb632f45 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:52.152287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:52.221927 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c130b8-7852-42e5-bc07-a53bdb632f45 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:52.254534 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da1586-27ed-4fd5-835a-9c758eeb0b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:52.255166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da1586-27ed-4fd5-835a-9c758eeb0b67 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:52.255447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da1586-27ed-4fd5-835a-9c758eeb0b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:52.255721 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da1586-27ed-4fd5-835a-9c758eeb0b67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:52.256540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0da1586-27ed-4fd5-835a-9c758eeb0b67 None admin] RBAC: Adding query filter params (role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f, scope.project.id=396f54b3e5f54e83a0ed520cb04cb77b, user.id=92ad46c03cf94f4682fe29e3c3684917) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:52.256848 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0da1586-27ed-4fd5-835a-9c758eeb0b67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:52.258264 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0da1586-27ed-4fd5-835a-9c758eeb0b67 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:52.263701 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 578/1185] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:28:52 2022] GET /identity/v3/role_assignments?user.id=92ad46c03cf94f4682fe29e3c3684917&scope.project.id=396f54b3e5f54e83a0ed520cb04cb77b&role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:54.263097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6acb0c-9028-4186-b5fb-4ebb54b8e227 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:54.263639 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6acb0c-9028-4186-b5fb-4ebb54b8e227 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:54.263639 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6acb0c-9028-4186-b5fb-4ebb54b8e227 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:54.265007 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 608/1186] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:28:54.270248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057e257-2d9d-497b-bf93-adda17d7bb97 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:54.270692 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057e257-2d9d-497b-bf93-adda17d7bb97 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:54.270851 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057e257-2d9d-497b-bf93-adda17d7bb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:54.287222 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-3057e257-2d9d-497b-bf93-adda17d7bb97 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:54.288393 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-3057e257-2d9d-497b-bf93-adda17d7bb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:54.353519 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 579/1187] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 18:28:54.376966 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6acb0c-9028-4186-b5fb-4ebb54b8e227 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:54.454932 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:54.473958 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6acb0c-9028-4186-b5fb-4ebb54b8e227 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:54.507039 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c2703-8ab7-40fc-8c5d-ad5fd617f9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:54.507801 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c2703-8ab7-40fc-8c5d-ad5fd617f9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:54.508082 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c2703-8ab7-40fc-8c5d-ad5fd617f9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:54.508390 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c2703-8ab7-40fc-8c5d-ad5fd617f9f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:54.518227 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c2703-8ab7-40fc-8c5d-ad5fd617f9f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:28:54.526178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-ab9c2703-8ab7-40fc-8c5d-ad5fd617f9f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:28:54.526178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:54.526178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:54.526178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:54.526178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:54.526178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:54.526178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:54.526178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:54.526178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:54.526178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:54.526178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:54.526178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:54.526178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:28:54.526178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:28:54.526178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:28:54.526178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:28:54.527301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:28:54.527301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:28:54.527301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:28:54.527301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:28:54.527301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:54.527301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:54.527301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:54.527301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:54.527301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:54.527301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:54.527301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:54.527301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:54.527301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:54.527301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:54.527301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:54.527301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:54.528560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:54.528560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:54.528560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:54.528560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:54.528560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:28:54.528560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:28:54.528560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:28:54.528560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:28:54.528560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:28:54.528560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:28:54.528560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:28:54.528560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:28:54.528560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 609/1188] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:28:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:28:54.533720 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057e257-2d9d-497b-bf93-adda17d7bb97 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:54.557254 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:54.630332 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057e257-2d9d-497b-bf93-adda17d7bb97 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:54.663215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b13c5-c67d-4967-89fc-572866676e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:54.663865 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b13c5-c67d-4967-89fc-572866676e91 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:54.664147 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b13c5-c67d-4967-89fc-572866676e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:54.664424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b13c5-c67d-4967-89fc-572866676e91 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:54.665238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b13c5-c67d-4967-89fc-572866676e91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:54.665541 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b13c5-c67d-4967-89fc-572866676e91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:54.666924 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b13c5-c67d-4967-89fc-572866676e91 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:54.674733 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 580/1189] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:28:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:54.680973 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c2703-8ab7-40fc-8c5d-ad5fd617f9f2 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:54.778053 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c2703-8ab7-40fc-8c5d-ad5fd617f9f2 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:54.810077 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ac30e-92fb-4693-bd3c-c5262756a605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:54.810956 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ac30e-92fb-4693-bd3c-c5262756a605 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:54.811237 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ac30e-92fb-4693-bd3c-c5262756a605 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:54.811511 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ac30e-92fb-4693-bd3c-c5262756a605 None admin] PATH_INFO: `/v3/users/92ad46c03cf94f4682fe29e3c3684917` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:54.812674 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7ac30e-92fb-4693-bd3c-c5262756a605 None admin] RBAC: Authorizing `identity:get_user(user_id=92ad46c03cf94f4682fe29e3c3684917)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:54.814100 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7ac30e-92fb-4693-bd3c-c5262756a605 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:54.815745 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 610/1190] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:28:54 2022] GET /identity/v3/users/92ad46c03cf94f4682fe29e3c3684917 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:54.822136 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b13c5-c67d-4967-89fc-572866676e91 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:54.905262 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:54.920216 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b13c5-c67d-4967-89fc-572866676e91 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:54.953283 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350d282-75e0-4106-a07a-cd26ff86aa44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:54.954122 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350d282-75e0-4106-a07a-cd26ff86aa44 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:54.954427 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350d282-75e0-4106-a07a-cd26ff86aa44 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:54.954706 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350d282-75e0-4106-a07a-cd26ff86aa44 None admin] PATH_INFO: `/v3/projects/396f54b3e5f54e83a0ed520cb04cb77b` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:54.955876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350d282-75e0-4106-a07a-cd26ff86aa44 None admin] RBAC: Authorizing `identity:get_project(project_id=396f54b3e5f54e83a0ed520cb04cb77b)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:54.957297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350d282-75e0-4106-a07a-cd26ff86aa44 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:54.959011 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 581/1191] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:28:54 2022] GET /identity/v3/projects/396f54b3e5f54e83a0ed520cb04cb77b => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:54.965427 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ac30e-92fb-4693-bd3c-c5262756a605 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:54.976419 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:55.061003 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ac30e-92fb-4693-bd3c-c5262756a605 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:55.092804 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29cf07-344e-47e6-897e-943956527e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:55.093657 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29cf07-344e-47e6-897e-943956527e6e None admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:55.093936 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29cf07-344e-47e6-897e-943956527e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:55.094206 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29cf07-344e-47e6-897e-943956527e6e None admin] PATH_INFO: `/v3/projects/396f54b3e5f54e83a0ed520cb04cb77b/users/92ad46c03cf94f4682fe29e3c3684917/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:55.096139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29cf07-344e-47e6-897e-943956527e6e None admin] RBAC: Authorizing `identity:create_grant(project_id=396f54b3e5f54e83a0ed520cb04cb77b, user_id=92ad46c03cf94f4682fe29e3c3684917, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:55.097548 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29cf07-344e-47e6-897e-943956527e6e None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:55.155503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 611/1192] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:28:54 2022] PUT /identity/v3/projects/396f54b3e5f54e83a0ed520cb04cb77b/users/92ad46c03cf94f4682fe29e3c3684917/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:28:57.161531 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab2bdc-81c9-4e0e-aa30-c797931a53a6 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:57.162078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab2bdc-81c9-4e0e-aa30-c797931a53a6 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:57.162078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab2bdc-81c9-4e0e-aa30-c797931a53a6 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:57.163481 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 582/1193] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:28:57.169243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e2651-1961-4790-8ea3-d04d4a0ede22 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:57.169641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e2651-1961-4790-8ea3-d04d4a0ede22 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:57.169803 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e2651-1961-4790-8ea3-d04d4a0ede22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:57.186776 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-be4e2651-1961-4790-8ea3-d04d4a0ede22 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:28:57.187953 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-be4e2651-1961-4790-8ea3-d04d4a0ede22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:28:57.247544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:57.252795 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 612/1194] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:28:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:28:57.276419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab2bdc-81c9-4e0e-aa30-c797931a53a6 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:57.377546 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab2bdc-81c9-4e0e-aa30-c797931a53a6 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:57.409640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4faa437-1867-4db2-9a58-ed6a87b4e060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:57.410512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4faa437-1867-4db2-9a58-ed6a87b4e060 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:57.410791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4faa437-1867-4db2-9a58-ed6a87b4e060 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:57.411062 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4faa437-1867-4db2-9a58-ed6a87b4e060 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:57.421344 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4faa437-1867-4db2-9a58-ed6a87b4e060 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:28:57.428724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-d4faa437-1867-4db2-9a58-ed6a87b4e060 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:28:57.428724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:28:57.428724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:28:57.428724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:28:57.428724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:28:57.428724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:28:57.428724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:28:57.428724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:28:57.428724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:28:57.428724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:28:57.428724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:28:57.428724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:28:57.428724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:28:57.428724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:28:57.428724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:28:57.428724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:28:57.429807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:28:57.429807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:28:57.429807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:28:57.429807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:28:57.429807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:28:57.429807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:28:57.429807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:28:57.429807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:28:57.429807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:28:57.429807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:28:57.429807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:28:57.429807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:28:57.429807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:28:57.429807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:28:57.429807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:28:57.429807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:28:57.430949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:28:57.430949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:28:57.430949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:28:57.430949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:28:57.430949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:28:57.430949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:28:57.430949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:28:57.430949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:28:57.430949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:28:57.430949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:28:57.430949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:28:57.430949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:28:57.430949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 583/1195] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:28:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:28:57.436008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e2651-1961-4790-8ea3-d04d4a0ede22 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:57.533024 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e2651-1961-4790-8ea3-d04d4a0ede22 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:57.566351 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406da6f9-0297-4bca-bfc4-8addc3063bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:57.567021 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406da6f9-0297-4bca-bfc4-8addc3063bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:57.567298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406da6f9-0297-4bca-bfc4-8addc3063bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:57.567569 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406da6f9-0297-4bca-bfc4-8addc3063bd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:57.568300 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406da6f9-0297-4bca-bfc4-8addc3063bd6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:57.568601 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406da6f9-0297-4bca-bfc4-8addc3063bd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:57.569959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406da6f9-0297-4bca-bfc4-8addc3063bd6 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:57.578102 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 613/1196] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:28:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:28:57.583739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4faa437-1867-4db2-9a58-ed6a87b4e060 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:57.599289 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:57.682115 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4faa437-1867-4db2-9a58-ed6a87b4e060 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:57.714016 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853be71-32b1-4cb5-b257-f3eb3542a928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:57.714881 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853be71-32b1-4cb5-b257-f3eb3542a928 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:57.715159 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853be71-32b1-4cb5-b257-f3eb3542a928 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:57.715430 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853be71-32b1-4cb5-b257-f3eb3542a928 None admin] PATH_INFO: `/v3/users/92ad46c03cf94f4682fe29e3c3684917` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:57.716590 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853be71-32b1-4cb5-b257-f3eb3542a928 None admin] RBAC: Authorizing `identity:get_user(user_id=92ad46c03cf94f4682fe29e3c3684917)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:57.717993 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853be71-32b1-4cb5-b257-f3eb3542a928 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:57.719624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 584/1197] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Jun 27 18:28:57 2022] GET /identity/v3/users/92ad46c03cf94f4682fe29e3c3684917 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:28:57.725223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406da6f9-0297-4bca-bfc4-8addc3063bd6 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:57.784209 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:57.824322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406da6f9-0297-4bca-bfc4-8addc3063bd6 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:57.856971 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d46983e-d94f-44d1-baec-8958c677f9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:57.857601 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46983e-d94f-44d1-baec-8958c677f9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:57.857890 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46983e-d94f-44d1-baec-8958c677f9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:57.858163 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46983e-d94f-44d1-baec-8958c677f9b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:57.922828 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d46983e-d94f-44d1-baec-8958c677f9b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:57.924268 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d46983e-d94f-44d1-baec-8958c677f9b9 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:57.979599 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:58.019618 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 614/1198] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:28:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 18:28:58.025438 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853be71-32b1-4cb5-b257-f3eb3542a928 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:58.103935 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:28:58.122603 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853be71-32b1-4cb5-b257-f3eb3542a928 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:58.155427 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0e1af-ea25-47cd-b094-abae10a2d2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:58.156261 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0e1af-ea25-47cd-b094-abae10a2d2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:58.156543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0e1af-ea25-47cd-b094-abae10a2d2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:58.156816 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0e1af-ea25-47cd-b094-abae10a2d2a5 None admin] PATH_INFO: `/v3/projects/396f54b3e5f54e83a0ed520cb04cb77b` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:58.157969 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0e1af-ea25-47cd-b094-abae10a2d2a5 None admin] RBAC: Authorizing `identity:get_project(project_id=396f54b3e5f54e83a0ed520cb04cb77b)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:58.159425 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0e1af-ea25-47cd-b094-abae10a2d2a5 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:58.161091 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 585/1199] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Jun 27 18:28:58 2022] GET /identity/v3/projects/396f54b3e5f54e83a0ed520cb04cb77b => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:28:58.167616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d46983e-d94f-44d1-baec-8958c677f9b9 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:28:58.265111 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d46983e-d94f-44d1-baec-8958c677f9b9 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:28:58.298680 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c21c38-baee-463f-b22d-2c90f519b148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:28:58.299316 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c21c38-baee-463f-b22d-2c90f519b148 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:28:58.299599 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c21c38-baee-463f-b22d-2c90f519b148 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:28:58.299875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c21c38-baee-463f-b22d-2c90f519b148 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:28:58.300695 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c21c38-baee-463f-b22d-2c90f519b148 None admin] RBAC: Adding query filter params (role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f, scope.project.id=396f54b3e5f54e83a0ed520cb04cb77b, user.id=92ad46c03cf94f4682fe29e3c3684917) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:28:58.301008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c21c38-baee-463f-b22d-2c90f519b148 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:28:58.302450 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c21c38-baee-463f-b22d-2c90f519b148 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:28:58.308422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 615/1200] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:28:58 2022] GET /identity/v3/role_assignments?user.id=92ad46c03cf94f4682fe29e3c3684917&scope.project.id=396f54b3e5f54e83a0ed520cb04cb77b&role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:29:08.151392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=26 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:29:08.305628 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=88 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:10.072381 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3918e-c4dc-4c8b-b2f2-0e5e8abdc595 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:10.072940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3918e-c4dc-4c8b-b2f2-0e5e8abdc595 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:10.072940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3918e-c4dc-4c8b-b2f2-0e5e8abdc595 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:10.074202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 586/1201] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:30:10.079310 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aeb484-886a-4548-a76c-ff536185165b None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:10.079756 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aeb484-886a-4548-a76c-ff536185165b None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:10.079862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aeb484-886a-4548-a76c-ff536185165b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:10.096654 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-89aeb484-886a-4548-a76c-ff536185165b None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:30:10.097897 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-89aeb484-886a-4548-a76c-ff536185165b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:30:10.166574 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 616/1202] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:30:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 18:30:10.189978 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f3918e-c4dc-4c8b-b2f2-0e5e8abdc595 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:10.289647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f3918e-c4dc-4c8b-b2f2-0e5e8abdc595 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:10.335909 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d30dd-788a-4d7a-8d79-97acc8b2bb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:10.335909 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d30dd-788a-4d7a-8d79-97acc8b2bb0f None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:10.335909 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d30dd-788a-4d7a-8d79-97acc8b2bb0f None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:10.335909 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d30dd-788a-4d7a-8d79-97acc8b2bb0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:10.340595 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d30dd-788a-4d7a-8d79-97acc8b2bb0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:10.342017 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d30dd-788a-4d7a-8d79-97acc8b2bb0f None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:10.359523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-788d30dd-788a-4d7a-8d79-97acc8b2bb0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:10.359523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:10.359523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:10.359523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:10.359523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:10.359523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:10.359523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:10.359523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:10.359523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:10.359523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:10.359523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:10.359523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:10.359523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:30:10.359523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:30:10.359523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:30:10.360723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:30:10.360723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:10.360723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:10.360723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:10.360723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:10.360723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:10.360723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:10.360723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:10.360723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:10.360723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:10.360723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:10.360723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:10.360723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:10.360723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:10.360723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:10.360723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:10.360723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:10.362022 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:30:10.362022 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:30:10.362022 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:10.362022 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:30:10.362022 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 587/1203] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:30:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:10.366780 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aeb484-886a-4548-a76c-ff536185165b None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:10.464724 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aeb484-886a-4548-a76c-ff536185165b None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:10.495720 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:10.501542 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb926cf4-3a39-4828-97fc-f112c158bd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:10.502193 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb926cf4-3a39-4828-97fc-f112c158bd4b None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:10.502507 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb926cf4-3a39-4828-97fc-f112c158bd4b None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:10.502788 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb926cf4-3a39-4828-97fc-f112c158bd4b None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:10.503508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb926cf4-3a39-4828-97fc-f112c158bd4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:10.503881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb926cf4-3a39-4828-97fc-f112c158bd4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:10.505376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb926cf4-3a39-4828-97fc-f112c158bd4b None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:10.516607 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 617/1204] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:10.522400 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d30dd-788a-4d7a-8d79-97acc8b2bb0f None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:10.575222 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:10.620288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d30dd-788a-4d7a-8d79-97acc8b2bb0f None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:10.652925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c5d66-b49d-47b2-8524-91634da60eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:10.654467 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c5d66-b49d-47b2-8524-91634da60eac None admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:10.654467 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c5d66-b49d-47b2-8524-91634da60eac None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:10.654467 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c5d66-b49d-47b2-8524-91634da60eac None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:10.654825 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c5d66-b49d-47b2-8524-91634da60eac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:10.656246 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c5d66-b49d-47b2-8524-91634da60eac None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:10.778940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 588/1205] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Jun 27 18:30:10 2022] POST /identity/v3/users => generated 251 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:30:12.850542 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91127ed2-9f0a-4764-8931-bafb5636d9c1 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:12.851091 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91127ed2-9f0a-4764-8931-bafb5636d9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:12.851091 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91127ed2-9f0a-4764-8931-bafb5636d9c1 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:12.852862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 618/1206] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:30:12.857923 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505741e-fe31-4011-b765-6ffa547246ca None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:12.858348 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505741e-fe31-4011-b765-6ffa547246ca None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:12.858510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505741e-fe31-4011-b765-6ffa547246ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:12.875184 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-4505741e-fe31-4011-b765-6ffa547246ca None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:30:12.876353 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-4505741e-fe31-4011-b765-6ffa547246ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:30:12.941687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 589/1207] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:30:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:30:12.966363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91127ed2-9f0a-4764-8931-bafb5636d9c1 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:13.067699 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91127ed2-9f0a-4764-8931-bafb5636d9c1 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:13.101025 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d857e-2fb7-4914-b428-ede26ff42ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:13.102228 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d857e-2fb7-4914-b428-ede26ff42ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:13.102795 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d857e-2fb7-4914-b428-ede26ff42ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:13.103070 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d857e-2fb7-4914-b428-ede26ff42ac1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:13.113073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d857e-2fb7-4914-b428-ede26ff42ac1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:30:13.121685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-2c2d857e-2fb7-4914-b428-ede26ff42ac1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:30:13.121685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:13.121685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:13.121685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:13.121685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:13.121685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:13.121685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:13.121685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:13.121685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:13.121685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:13.121685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:13.121685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:13.121685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:30:13.121685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:30:13.121685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:30:13.121685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:30:13.122826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:30:13.122826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:30:13.122826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:30:13.122826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:30:13.122826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:13.122826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:13.122826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:13.122826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:13.122826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:13.122826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:13.122826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:13.122826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:13.122826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:13.122826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:13.122826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:13.122826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:13.123894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:13.123894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:13.123894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:13.123894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:13.123894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:30:13.123894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:30:13.123894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:30:13.123894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:30:13.123894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:30:13.123894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:30:13.123894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:30:13.123894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:13.126537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 619/1208] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:30:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:13.128769 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505741e-fe31-4011-b765-6ffa547246ca None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:13.167022 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:13.223885 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505741e-fe31-4011-b765-6ffa547246ca None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:13.256735 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce04b3f-2736-426b-b579-3fa9a1edd0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:13.257378 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce04b3f-2736-426b-b579-3fa9a1edd0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:13.257679 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce04b3f-2736-426b-b579-3fa9a1edd0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:13.257945 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce04b3f-2736-426b-b579-3fa9a1edd0e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:13.258702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce04b3f-2736-426b-b579-3fa9a1edd0e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:13.259001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce04b3f-2736-426b-b579-3fa9a1edd0e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:13.260343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce04b3f-2736-426b-b579-3fa9a1edd0e4 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:13.267804 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 590/1209] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Jun 27 18:30:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:13.274090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d857e-2fb7-4914-b428-ede26ff42ac1 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:13.324252 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:13.374929 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d857e-2fb7-4914-b428-ede26ff42ac1 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:13.407998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f377d20-39f4-453e-a6c9-7db49f6b3159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:13.408770 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f377d20-39f4-453e-a6c9-7db49f6b3159 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:13.409049 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f377d20-39f4-453e-a6c9-7db49f6b3159 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:13.409320 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f377d20-39f4-453e-a6c9-7db49f6b3159 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:13.419752 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f377d20-39f4-453e-a6c9-7db49f6b3159 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:13.421171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f377d20-39f4-453e-a6c9-7db49f6b3159 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:13.438870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-1f377d20-39f4-453e-a6c9-7db49f6b3159 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:13.438870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:13.438870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:13.438870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:13.438870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:13.438870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:13.438870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:13.438870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:13.438870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:13.438870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:13.438870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:13.438870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:13.438870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:30:13.438870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:30:13.438870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:30:13.439940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:30:13.439940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:13.439940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:13.439940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:13.439940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:13.439940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:13.439940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:13.439940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:13.439940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:13.439940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:13.439940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:13.439940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:13.439940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:13.439940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:13.439940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:13.439940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:13.439940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:13.441305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:30:13.441305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:30:13.441305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:13.441305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:13.441305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 620/1210] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:30:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:13.446203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce04b3f-2736-426b-b579-3fa9a1edd0e4 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:13.543096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:13.545938 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce04b3f-2736-426b-b579-3fa9a1edd0e4 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:13.578868 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a3c48-08c4-4426-8f27-6242d7e2dc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:13.579519 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a3c48-08c4-4426-8f27-6242d7e2dc8c None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:13.579799 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a3c48-08c4-4426-8f27-6242d7e2dc8c None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:13.580071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a3c48-08c4-4426-8f27-6242d7e2dc8c None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:13.580814 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a3c48-08c4-4426-8f27-6242d7e2dc8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:13.581118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a3c48-08c4-4426-8f27-6242d7e2dc8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:13.582489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a3c48-08c4-4426-8f27-6242d7e2dc8c None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:13.593876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 591/1211] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:13.599656 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f377d20-39f4-453e-a6c9-7db49f6b3159 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:13.698452 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f377d20-39f4-453e-a6c9-7db49f6b3159 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:13.699939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:13.733083 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75d19e-aea0-4b00-96dc-309b421b6a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:13.733867 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75d19e-aea0-4b00-96dc-309b421b6a14 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:13.734147 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75d19e-aea0-4b00-96dc-309b421b6a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:13.734442 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75d19e-aea0-4b00-96dc-309b421b6a14 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:13.741585 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75d19e-aea0-4b00-96dc-309b421b6a14 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:13.743841 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75d19e-aea0-4b00-96dc-309b421b6a14 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:13.758777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-8b75d19e-aea0-4b00-96dc-309b421b6a14 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 18:30:13.758777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:13.758777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:13.758777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:13.758777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:13.758777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:13.758777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:13.758777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:13.758777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:13.758777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:13.758777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:13.758777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:13.758777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:30:13.758777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:30:13.758777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:30:13.758777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:30:13.759768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:13.759768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:13.759768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:30:13.759768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:30:13.759768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:30:13.759768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:30:13.759768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:13.759768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:13.759768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:13.759768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:13.759768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:13.759768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:13.759768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:13.759768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:13.759768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:13.759768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:13.759768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:13.759768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:13.760982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:13.760982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:13.760982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:30:13.760982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:30:13.760982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:30:13.760982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:30:13.760982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:30:13.760982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:30:13.760982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 18:30:13.760982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:13.760982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 621/1212] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:30:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:13.765914 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a3c48-08c4-4426-8f27-6242d7e2dc8c None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:13.865525 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a3c48-08c4-4426-8f27-6242d7e2dc8c None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:13.898020 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ddaeba-bf86-470e-9f0c-812478f7e910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:13.899231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddaeba-bf86-470e-9f0c-812478f7e910 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:13.899516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddaeba-bf86-470e-9f0c-812478f7e910 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:13.899789 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddaeba-bf86-470e-9f0c-812478f7e910 None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:13.900609 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ddaeba-bf86-470e-9f0c-812478f7e910 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:13.900915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ddaeba-bf86-470e-9f0c-812478f7e910 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:13.902440 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ddaeba-bf86-470e-9f0c-812478f7e910 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:13.919974 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 592/1213] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Jun 27 18:30:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:13.925940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75d19e-aea0-4b00-96dc-309b421b6a14 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:14.024201 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75d19e-aea0-4b00-96dc-309b421b6a14 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:14.057123 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf47b6c-5239-439f-97f6-a655673112fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:14.057737 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf47b6c-5239-439f-97f6-a655673112fb None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:14.058015 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf47b6c-5239-439f-97f6-a655673112fb None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:14.058284 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf47b6c-5239-439f-97f6-a655673112fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:14.090007 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:14.126682 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf47b6c-5239-439f-97f6-a655673112fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:14.128073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf47b6c-5239-439f-97f6-a655673112fb None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:14.226322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 622/1214] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:30:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 18:30:14.232678 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ddaeba-bf86-470e-9f0c-812478f7e910 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:14.251381 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:14.332439 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ddaeba-bf86-470e-9f0c-812478f7e910 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:14.366290 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49798b92-ea48-4f06-9a5e-c00f53016548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:14.367129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49798b92-ea48-4f06-9a5e-c00f53016548 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:14.367634 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49798b92-ea48-4f06-9a5e-c00f53016548 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:14.367740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49798b92-ea48-4f06-9a5e-c00f53016548 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:14.381331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49798b92-ea48-4f06-9a5e-c00f53016548 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:14.381331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49798b92-ea48-4f06-9a5e-c00f53016548 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:14.396708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-49798b92-ea48-4f06-9a5e-c00f53016548 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:14.396708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:14.396708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:14.396708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:14.396708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:14.396708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:14.396708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:14.396708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:14.396708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:14.396708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:14.396708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:14.396708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:14.396708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:30:14.396708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:30:14.396708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:30:14.397780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:30:14.397780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:14.397780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:14.397780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:14.397780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:14.397780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:14.397780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:14.397780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:14.397780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:14.397780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:14.397780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:14.397780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:14.397780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:14.397780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:14.397780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:14.397780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:14.397780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:14.399624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:30:14.399624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:30:14.399624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:14.399624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:30:14.399624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 593/1215] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:30:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:14.404962 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf47b6c-5239-439f-97f6-a655673112fb None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:14.465362 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:14.507089 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf47b6c-5239-439f-97f6-a655673112fb None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:14.540713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f653b-7f35-487c-b24e-c937dac2bf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:14.541365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f653b-7f35-487c-b24e-c937dac2bf7d None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:14.541648 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f653b-7f35-487c-b24e-c937dac2bf7d None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:14.541922 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f653b-7f35-487c-b24e-c937dac2bf7d None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:14.542671 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f653b-7f35-487c-b24e-c937dac2bf7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:14.542976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f653b-7f35-487c-b24e-c937dac2bf7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:14.544342 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f653b-7f35-487c-b24e-c937dac2bf7d None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:14.555696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 623/1216] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:14.561391 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49798b92-ea48-4f06-9a5e-c00f53016548 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:14.630422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:14.661019 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49798b92-ea48-4f06-9a5e-c00f53016548 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:14.696828 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9aea05-78cd-472c-bd3e-f9d83d287384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:14.697651 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9aea05-78cd-472c-bd3e-f9d83d287384 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:14.697938 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9aea05-78cd-472c-bd3e-f9d83d287384 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:14.698215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9aea05-78cd-472c-bd3e-f9d83d287384 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:14.704907 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9aea05-78cd-472c-bd3e-f9d83d287384 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:14.708556 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9aea05-78cd-472c-bd3e-f9d83d287384 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:14.720637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-ce9aea05-78cd-472c-bd3e-f9d83d287384 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:30:14.720637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:14.720637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:14.720637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:14.720637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:14.720637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:14.720637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:14.720637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:14.720637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:14.720637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:14.720637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:14.720637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:14.720637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:30:14.720637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:30:14.720637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:30:14.721672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:30:14.721672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:14.721672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:14.721672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:14.721672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:14.721672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:14.721672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:14.721672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:14.721672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:14.721672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:14.721672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:14.721672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:14.721672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:14.721672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:14.721672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:14.721672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:14.721672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:14.722834 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:30:14.722834 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:30:14.722834 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:30:14.722834 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:30:14.722834 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:30:14.722834 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:30:14.722834 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:30:14.722834 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:30:14.722834 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 594/1217] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:30:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:30:14.730396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f653b-7f35-487c-b24e-c937dac2bf7d None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:14.830151 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f653b-7f35-487c-b24e-c937dac2bf7d None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:14.853711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:14.864694 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1eb210-7b89-4a91-ad4e-99f1bf05bef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:14.865338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1eb210-7b89-4a91-ad4e-99f1bf05bef3 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:14.865614 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1eb210-7b89-4a91-ad4e-99f1bf05bef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:14.865884 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1eb210-7b89-4a91-ad4e-99f1bf05bef3 None admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:14.866698 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1eb210-7b89-4a91-ad4e-99f1bf05bef3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:14.867172 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1eb210-7b89-4a91-ad4e-99f1bf05bef3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:14.868681 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1eb210-7b89-4a91-ad4e-99f1bf05bef3 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:14.880392 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 624/1218] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:30:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:14.886991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9aea05-78cd-472c-bd3e-f9d83d287384 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:14.987326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9aea05-78cd-472c-bd3e-f9d83d287384 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:15.014970 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:15.020347 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86deae-9431-4043-aa93-7c6560a41ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:15.020982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86deae-9431-4043-aa93-7c6560a41ebc None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:15.021262 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86deae-9431-4043-aa93-7c6560a41ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:15.021535 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86deae-9431-4043-aa93-7c6560a41ebc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:15.022344 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86deae-9431-4043-aa93-7c6560a41ebc None admin] RBAC: Adding query filter params (role.id=c4bc6e72b9d647c5b2485d52d55f7dd4, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=3cd6f0a1286243e4add692db01028205) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:15.022680 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86deae-9431-4043-aa93-7c6560a41ebc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:15.025001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86deae-9431-4043-aa93-7c6560a41ebc None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:15.030266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 595/1219] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:30:14 2022] GET /identity/v3/role_assignments?user.id=3cd6f0a1286243e4add692db01028205&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=c4bc6e72b9d647c5b2485d52d55f7dd4 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:17.067947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900a775-3ea8-4c43-a286-30861b79b23e None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:17.068451 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900a775-3ea8-4c43-a286-30861b79b23e None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:17.068451 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900a775-3ea8-4c43-a286-30861b79b23e None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:17.069801 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 625/1220] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:30:17.075110 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecab0ed-35ab-404e-a3dc-b4fb64d00cce None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:17.075535 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecab0ed-35ab-404e-a3dc-b4fb64d00cce None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:17.075668 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecab0ed-35ab-404e-a3dc-b4fb64d00cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:17.093358 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-8ecab0ed-35ab-404e-a3dc-b4fb64d00cce None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:30:17.094582 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-8ecab0ed-35ab-404e-a3dc-b4fb64d00cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:30:17.160625 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 596/1221] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:30:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:30:17.183219 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900a775-3ea8-4c43-a286-30861b79b23e None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:17.283077 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900a775-3ea8-4c43-a286-30861b79b23e None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:17.315665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef0bfb-e474-42a4-90f0-ee919c32157a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:17.316439 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef0bfb-e474-42a4-90f0-ee919c32157a None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:17.316759 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef0bfb-e474-42a4-90f0-ee919c32157a None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:17.317031 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef0bfb-e474-42a4-90f0-ee919c32157a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:17.328177 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef0bfb-e474-42a4-90f0-ee919c32157a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:30:17.334959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-adef0bfb-e474-42a4-90f0-ee919c32157a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:30:17.334959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:17.334959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:17.334959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:17.334959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:17.334959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:17.334959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:17.334959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:17.334959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:17.334959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:17.334959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:17.334959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:17.334959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:30:17.334959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:30:17.334959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:30:17.334959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:30:17.336719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:30:17.336719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:30:17.336719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:30:17.336719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:30:17.336719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:17.336719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:17.336719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:17.336719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:17.336719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:17.336719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:17.336719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:17.336719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:17.336719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:17.336719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:17.336719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:17.336719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:17.337691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:17.337691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:17.337691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:17.337691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:17.337691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:30:17.337691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:30:17.337691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:30:17.337691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:30:17.337691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:30:17.337691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:30:17.337691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:30:17.337691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:17.337691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 626/1222] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:30:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:17.343052 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecab0ed-35ab-404e-a3dc-b4fb64d00cce None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:17.439027 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecab0ed-35ab-404e-a3dc-b4fb64d00cce None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:17.463540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:17.473726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de06b13-fd1a-42ea-b1a3-df6f0342f9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:17.474376 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de06b13-fd1a-42ea-b1a3-df6f0342f9b0 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:17.474692 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de06b13-fd1a-42ea-b1a3-df6f0342f9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:17.474966 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de06b13-fd1a-42ea-b1a3-df6f0342f9b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:17.475702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de06b13-fd1a-42ea-b1a3-df6f0342f9b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:17.476005 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de06b13-fd1a-42ea-b1a3-df6f0342f9b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:17.478557 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de06b13-fd1a-42ea-b1a3-df6f0342f9b0 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:17.486187 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 597/1223] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Jun 27 18:30:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:17.492599 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef0bfb-e474-42a4-90f0-ee919c32157a None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:17.536292 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:17.591291 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef0bfb-e474-42a4-90f0-ee919c32157a None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:17.624306 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8739d6-09ef-4ea0-9215-e086aed09e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:17.624961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8739d6-09ef-4ea0-9215-e086aed09e37 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:17.625244 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8739d6-09ef-4ea0-9215-e086aed09e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:17.625516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8739d6-09ef-4ea0-9215-e086aed09e37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:17.641304 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8739d6-09ef-4ea0-9215-e086aed09e37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:17.642851 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8739d6-09ef-4ea0-9215-e086aed09e37 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:17.660041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-0f8739d6-09ef-4ea0-9215-e086aed09e37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:17.660041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:17.660041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:17.660041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:17.660041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:17.660041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:17.660041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:17.660041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:17.660041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:17.660041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:17.660041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:17.660041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:17.660041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:30:17.660041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:30:17.660041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:30:17.661144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:30:17.661144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:17.661144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:17.661144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:17.661144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:17.661144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:17.661144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:17.661144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:17.661144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:17.661144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:17.661144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:17.661144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:17.661144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:17.661144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:17.661144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:17.661144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:17.661144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:17.662254 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:30:17.662254 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:30:17.662254 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:17.662254 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:17.662254 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 627/1224] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:30:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:17.667842 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de06b13-fd1a-42ea-b1a3-df6f0342f9b0 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:17.770511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de06b13-fd1a-42ea-b1a3-df6f0342f9b0 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:17.802623 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede66806-25e8-487c-9d08-6bc2d23b0ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:17.803286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede66806-25e8-487c-9d08-6bc2d23b0ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:17.803551 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede66806-25e8-487c-9d08-6bc2d23b0ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:17.803822 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede66806-25e8-487c-9d08-6bc2d23b0ca4 None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:17.805036 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede66806-25e8-487c-9d08-6bc2d23b0ca4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:17.805337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede66806-25e8-487c-9d08-6bc2d23b0ca4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:17.806702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede66806-25e8-487c-9d08-6bc2d23b0ca4 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:17.818089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 598/1225] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:30:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:17.824691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8739d6-09ef-4ea0-9215-e086aed09e37 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:17.917646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:17.924215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8739d6-09ef-4ea0-9215-e086aed09e37 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:17.956594 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c7d81-ec22-45ee-a3b8-95449e77b530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:17.957373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c7d81-ec22-45ee-a3b8-95449e77b530 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:17.957655 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c7d81-ec22-45ee-a3b8-95449e77b530 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:17.957928 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c7d81-ec22-45ee-a3b8-95449e77b530 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:17.965257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c7d81-ec22-45ee-a3b8-95449e77b530 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:17.966717 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c7d81-ec22-45ee-a3b8-95449e77b530 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:17.981186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-0a1c7d81-ec22-45ee-a3b8-95449e77b530 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 18:30:17.981186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:17.981186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:17.981186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:17.981186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:17.981186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:17.981186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:17.981186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:17.981186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:17.981186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:17.981186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:17.981186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:17.981186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:30:17.981186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:30:17.981186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:30:17.981186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:30:17.985969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:17.985969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:17.985969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:30:17.985969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:30:17.985969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:30:17.985969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:30:17.985969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:17.985969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:17.985969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:17.985969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:17.985969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:17.985969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:17.985969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:17.985969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:17.985969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:17.985969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:17.985969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:17.985969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:17.987222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:17.987222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:17.987222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:30:17.987222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:30:17.987222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:30:17.987222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:30:17.987222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:30:17.987222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:30:17.987222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 18:30:17.987222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:17.987222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 628/1226] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:30:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:17.988471 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede66806-25e8-487c-9d08-6bc2d23b0ca4 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:18.035999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:18.088409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede66806-25e8-487c-9d08-6bc2d23b0ca4 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:18.121946 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6bee74-5658-4397-b1b7-15dc70a96c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:18.122621 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bee74-5658-4397-b1b7-15dc70a96c44 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:18.122901 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bee74-5658-4397-b1b7-15dc70a96c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:18.123171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bee74-5658-4397-b1b7-15dc70a96c44 None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:18.123987 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6bee74-5658-4397-b1b7-15dc70a96c44 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:18.124287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6bee74-5658-4397-b1b7-15dc70a96c44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:18.125696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6bee74-5658-4397-b1b7-15dc70a96c44 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:18.143380 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 599/1227] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Jun 27 18:30:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:30:18.149498 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c7d81-ec22-45ee-a3b8-95449e77b530 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:18.250401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c7d81-ec22-45ee-a3b8-95449e77b530 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:18.284752 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673a7a4-b206-4e85-a9a7-040f0ade3fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:18.285537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673a7a4-b206-4e85-a9a7-040f0ade3fde None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:18.285822 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673a7a4-b206-4e85-a9a7-040f0ade3fde None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:18.286096 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673a7a4-b206-4e85-a9a7-040f0ade3fde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:18.297124 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673a7a4-b206-4e85-a9a7-040f0ade3fde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:18.298621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673a7a4-b206-4e85-a9a7-040f0ade3fde None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:18.315309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-b673a7a4-b206-4e85-a9a7-040f0ade3fde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:18.315309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:18.315309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:18.315309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:18.315309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:18.315309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:18.315309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:18.315309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:18.315309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:18.315309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:18.315309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:18.315309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:18.315309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:30:18.315309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:30:18.315309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:30:18.316475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:30:18.316475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:18.316475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:18.316475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:18.316475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:18.316475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:18.316475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:18.316475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:18.316475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:18.316475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:18.316475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:18.316475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:18.316475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:18.316475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:18.316475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:18.316475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:18.316475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:18.317783 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:30:18.317783 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:30:18.317783 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:18.317783 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:18.317783 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 629/1228] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:30:18.322562 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6bee74-5658-4397-b1b7-15dc70a96c44 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:18.422260 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6bee74-5658-4397-b1b7-15dc70a96c44 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:18.424556 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:18.455328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ee4a5-009e-4b63-bffc-52267881b614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:18.455972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ee4a5-009e-4b63-bffc-52267881b614 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:18.456254 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ee4a5-009e-4b63-bffc-52267881b614 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:18.456531 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ee4a5-009e-4b63-bffc-52267881b614 None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:18.457254 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ee4a5-009e-4b63-bffc-52267881b614 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:18.457563 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ee4a5-009e-4b63-bffc-52267881b614 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:18.459062 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ee4a5-009e-4b63-bffc-52267881b614 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:18.470781 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 600/1229] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:18.477148 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673a7a4-b206-4e85-a9a7-040f0ade3fde None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:18.486669 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:18.575487 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673a7a4-b206-4e85-a9a7-040f0ade3fde None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:18.613476 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364d9f7-98ed-480d-80ba-e5006ccb3a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:18.614270 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364d9f7-98ed-480d-80ba-e5006ccb3a80 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:18.614569 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364d9f7-98ed-480d-80ba-e5006ccb3a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:18.614843 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364d9f7-98ed-480d-80ba-e5006ccb3a80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:18.621652 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364d9f7-98ed-480d-80ba-e5006ccb3a80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:18.623123 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364d9f7-98ed-480d-80ba-e5006ccb3a80 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:18.637350 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-1364d9f7-98ed-480d-80ba-e5006ccb3a80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:30:18.637350 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:18.637350 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:18.637350 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:18.637350 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:18.637350 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:18.637350 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:18.637350 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:18.637350 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:18.637350 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:18.637350 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:18.637350 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:18.637350 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:30:18.637350 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:30:18.637350 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:30:18.638466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:30:18.638466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:18.638466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:18.638466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:18.638466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:18.638466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:18.638466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:18.638466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:18.638466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:18.638466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:18.638466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:18.638466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:18.638466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:18.638466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:18.638466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:18.638466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:18.638466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:18.639566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:30:18.639566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:30:18.639566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:30:18.639566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:30:18.639566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:30:18.639566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:30:18.639566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:30:18.639566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:18.639566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 630/1230] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:30:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:18.644383 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ee4a5-009e-4b63-bffc-52267881b614 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:18.744889 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ee4a5-009e-4b63-bffc-52267881b614 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:18.777321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcabb8ed-9d06-4638-a2ec-1ff9a7b059ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:18.778005 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcabb8ed-9d06-4638-a2ec-1ff9a7b059ac None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:18.778291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcabb8ed-9d06-4638-a2ec-1ff9a7b059ac None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:18.778643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcabb8ed-9d06-4638-a2ec-1ff9a7b059ac None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:18.779531 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcabb8ed-9d06-4638-a2ec-1ff9a7b059ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:18.779838 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcabb8ed-9d06-4638-a2ec-1ff9a7b059ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:18.781310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcabb8ed-9d06-4638-a2ec-1ff9a7b059ac None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:18.794050 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 601/1231] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:30:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:18.800201 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364d9f7-98ed-480d-80ba-e5006ccb3a80 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:18.865557 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:18.899401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364d9f7-98ed-480d-80ba-e5006ccb3a80 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:18.934710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ca033-30df-469b-9977-a0a1e7b77169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:18.935582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ca033-30df-469b-9977-a0a1e7b77169 None admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:18.935859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ca033-30df-469b-9977-a0a1e7b77169 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:18.936131 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ca033-30df-469b-9977-a0a1e7b77169 None admin] PATH_INFO: `/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/3cd6f0a1286243e4add692db01028205/roles/c4bc6e72b9d647c5b2485d52d55f7dd4` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:18.950730 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ca033-30df-469b-9977-a0a1e7b77169 None admin] RBAC: Authorizing `identity:create_grant(project_id=84ecf552f8364884b4aeff3428cb81e4, user_id=3cd6f0a1286243e4add692db01028205, role_id=c4bc6e72b9d647c5b2485d52d55f7dd4)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:18.952183 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ca033-30df-469b-9977-a0a1e7b77169 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:19.040245 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 631/1232] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:30:18 2022] PUT /identity/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/3cd6f0a1286243e4add692db01028205/roles/c4bc6e72b9d647c5b2485d52d55f7dd4 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:30:21.038002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41465b-c545-476f-a0ca-ef193aeeff9d None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:21.039040 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41465b-c545-476f-a0ca-ef193aeeff9d None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:21.039160 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41465b-c545-476f-a0ca-ef193aeeff9d None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:21.040498 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 602/1233] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:30:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:30:21.045915 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb619e9-b43f-440a-a46f-4f5ddafd7d7d None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:21.050888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb619e9-b43f-440a-a46f-4f5ddafd7d7d None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:21.050888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb619e9-b43f-440a-a46f-4f5ddafd7d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:21.063194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-ffb619e9-b43f-440a-a46f-4f5ddafd7d7d None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:30:21.064371 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ffb619e9-b43f-440a-a46f-4f5ddafd7d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:30:21.130598 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 632/1234] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:30:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:30:21.154090 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41465b-c545-476f-a0ca-ef193aeeff9d None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:21.180090 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:21.255666 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41465b-c545-476f-a0ca-ef193aeeff9d None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:21.290206 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f26cea-9328-4011-a473-509d1d1a1ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:21.290949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f26cea-9328-4011-a473-509d1d1a1ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:21.291230 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f26cea-9328-4011-a473-509d1d1a1ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:21.291500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f26cea-9328-4011-a473-509d1d1a1ca8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:21.301359 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f26cea-9328-4011-a473-509d1d1a1ca8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:30:21.309061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-f0f26cea-9328-4011-a473-509d1d1a1ca8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:30:21.309061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:21.309061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:21.309061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:21.309061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:21.309061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:21.309061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:21.309061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:21.309061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:21.309061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:21.309061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:21.309061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:21.309061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:30:21.309061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:30:21.309061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:30:21.309061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:30:21.310171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:30:21.310171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:30:21.310171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:30:21.310171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:30:21.310171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:21.310171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:21.310171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:21.310171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:21.310171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:21.310171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:21.310171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:21.310171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:21.310171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:21.310171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:21.310171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:21.310171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:21.311291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:21.311291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:21.311291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:21.311291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:21.311291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:30:21.311291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:30:21.311291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:30:21.311291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:30:21.311291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:30:21.311291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:30:21.311291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 18:30:21.311291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:30:21.311291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 603/1235] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Jun 27 18:30:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:21.316331 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb619e9-b43f-440a-a46f-4f5ddafd7d7d None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:21.363540 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:21.416081 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb619e9-b43f-440a-a46f-4f5ddafd7d7d None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:21.447866 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded7c09-6bb7-4b79-8d48-5fa708f57585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:21.448518 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded7c09-6bb7-4b79-8d48-5fa708f57585 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:21.448797 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded7c09-6bb7-4b79-8d48-5fa708f57585 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:21.449070 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded7c09-6bb7-4b79-8d48-5fa708f57585 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:21.449859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded7c09-6bb7-4b79-8d48-5fa708f57585 None admin] RBAC: Adding query filter params (name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:21.450162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded7c09-6bb7-4b79-8d48-5fa708f57585 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:21.451550 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded7c09-6bb7-4b79-8d48-5fa708f57585 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:21.459544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 633/1236] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Jun 27 18:30:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:21.465885 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f26cea-9328-4011-a473-509d1d1a1ca8 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:21.550939 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:21.566219 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f26cea-9328-4011-a473-509d1d1a1ca8 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:21.599505 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5184a-2b4f-461f-b14f-aa451402f1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:21.600280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5184a-2b4f-461f-b14f-aa451402f1f9 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:21.600564 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5184a-2b4f-461f-b14f-aa451402f1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:21.600863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5184a-2b4f-461f-b14f-aa451402f1f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:21.611545 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be5184a-2b4f-461f-b14f-aa451402f1f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:21.612977 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be5184a-2b4f-461f-b14f-aa451402f1f9 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:21.630439 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-1be5184a-2b4f-461f-b14f-aa451402f1f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:21.630439 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:21.630439 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:21.630439 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:21.630439 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:21.630439 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:21.630439 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:21.630439 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:21.630439 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:21.630439 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:21.630439 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:21.630439 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:21.630439 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:30:21.630439 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:30:21.630439 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:30:21.631510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:30:21.631510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:21.631510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:21.631510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:21.631510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:21.631510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:21.631510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:21.631510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:21.631510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:21.631510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:21.631510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:21.631510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:21.631510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:21.631510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:21.631510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:21.631510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:21.631510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:21.633326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:30:21.633326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:30:21.633326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:21.633326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:30:21.633326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 604/1237] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:30:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:21.637823 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded7c09-6bb7-4b79-8d48-5fa708f57585 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:21.736173 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded7c09-6bb7-4b79-8d48-5fa708f57585 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:21.745553 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:21.769996 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57372f-8716-4698-8481-c852c67b19f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:21.770923 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57372f-8716-4698-8481-c852c67b19f6 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:21.771369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57372f-8716-4698-8481-c852c67b19f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:21.771647 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57372f-8716-4698-8481-c852c67b19f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:21.772391 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57372f-8716-4698-8481-c852c67b19f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:21.772698 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57372f-8716-4698-8481-c852c67b19f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:21.774685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57372f-8716-4698-8481-c852c67b19f6 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:21.785994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 634/1238] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:21.791741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5184a-2b4f-461f-b14f-aa451402f1f9 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:21.890262 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5184a-2b4f-461f-b14f-aa451402f1f9 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:21.925044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d928f-0fe5-4097-867b-a4211ade956a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:21.925836 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d928f-0fe5-4097-867b-a4211ade956a None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:21.926119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d928f-0fe5-4097-867b-a4211ade956a None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:21.926408 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d928f-0fe5-4097-867b-a4211ade956a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:21.929089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:21.933609 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d928f-0fe5-4097-867b-a4211ade956a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:21.935075 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d928f-0fe5-4097-867b-a4211ade956a None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:21.949915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-356d928f-0fe5-4097-867b-a4211ade956a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 18:30:21.949915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:21.949915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:21.949915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:21.949915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:21.949915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:21.949915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:21.949915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:21.949915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:21.949915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:21.949915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:21.949915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:21.949915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:30:21.949915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:30:21.949915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:30:21.949915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:30:21.951092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:21.951092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:21.951092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:30:21.951092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:30:21.951092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:30:21.951092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:30:21.951092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:21.951092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:21.951092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:21.951092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:21.951092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:21.951092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:21.951092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:21.951092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:21.951092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:21.951092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:21.951092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:21.951092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:21.952096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:21.952096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:21.952096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:30:21.952096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:30:21.952096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:30:21.952096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:30:21.952096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:30:21.952096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:30:21.952096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 18:30:21.952096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:30:21.952096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 605/1239] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:30:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:21.956943 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57372f-8716-4698-8481-c852c67b19f6 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:22.056847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57372f-8716-4698-8481-c852c67b19f6 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:22.090727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e64287-123e-4c43-a3d9-49f725b35699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:22.091380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e64287-123e-4c43-a3d9-49f725b35699 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:22.091663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e64287-123e-4c43-a3d9-49f725b35699 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:22.091938 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e64287-123e-4c43-a3d9-49f725b35699 None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:22.092763 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e64287-123e-4c43-a3d9-49f725b35699 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:22.093067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e64287-123e-4c43-a3d9-49f725b35699 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:22.094504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e64287-123e-4c43-a3d9-49f725b35699 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:22.112431 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 635/1240] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Jun 27 18:30:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:22.118205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d928f-0fe5-4097-867b-a4211ade956a None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:22.217387 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d928f-0fe5-4097-867b-a4211ade956a None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:22.251670 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a490cb6-e5c7-43d5-b20d-f67a9e57cf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:22.252297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a490cb6-e5c7-43d5-b20d-f67a9e57cf27 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:22.252574 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a490cb6-e5c7-43d5-b20d-f67a9e57cf27 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:22.252853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a490cb6-e5c7-43d5-b20d-f67a9e57cf27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:22.313919 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:22.320576 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a490cb6-e5c7-43d5-b20d-f67a9e57cf27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:22.322142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a490cb6-e5c7-43d5-b20d-f67a9e57cf27 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:22.419391 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 606/1241] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:30:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 18:30:22.425357 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e64287-123e-4c43-a3d9-49f725b35699 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:22.452808 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:22.524358 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e64287-123e-4c43-a3d9-49f725b35699 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:22.559030 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37d5c3-53cf-4f33-9c2d-a474b586f24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:22.559813 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37d5c3-53cf-4f33-9c2d-a474b586f24c None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:22.560096 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37d5c3-53cf-4f33-9c2d-a474b586f24c None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:22.560371 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37d5c3-53cf-4f33-9c2d-a474b586f24c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:22.570849 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37d5c3-53cf-4f33-9c2d-a474b586f24c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:22.572279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37d5c3-53cf-4f33-9c2d-a474b586f24c None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:22.590747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-9f37d5c3-53cf-4f33-9c2d-a474b586f24c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:22.590747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:22.590747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:22.590747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:22.590747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:22.590747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:22.590747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:22.590747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:22.590747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:22.590747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:22.590747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:22.590747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:22.590747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:30:22.590747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:30:22.590747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:30:22.593460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:30:22.593460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:22.593460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:22.593460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:22.593460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:22.593460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:22.593460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:22.593460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:22.593460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:22.593460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:22.593460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:22.593460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:22.593460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:22.593460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:22.593460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:22.593460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:22.593460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:22.594442 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:30:22.594442 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:30:22.594442 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:22.594442 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:22.594442 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 636/1242] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:30:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 18:30:22.597716 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a490cb6-e5c7-43d5-b20d-f67a9e57cf27 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:22.688925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:22.695060 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a490cb6-e5c7-43d5-b20d-f67a9e57cf27 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:22.728724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692583b6-b4f5-4471-8195-928f4e8739ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:22.729378 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692583b6-b4f5-4471-8195-928f4e8739ac None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:22.729674 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692583b6-b4f5-4471-8195-928f4e8739ac None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:22.729932 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692583b6-b4f5-4471-8195-928f4e8739ac None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:22.730686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692583b6-b4f5-4471-8195-928f4e8739ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:22.730990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692583b6-b4f5-4471-8195-928f4e8739ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:22.732354 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692583b6-b4f5-4471-8195-928f4e8739ac None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:22.744350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 607/1243] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:22.749900 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37d5c3-53cf-4f33-9c2d-a474b586f24c None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:22.827715 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:22.847849 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37d5c3-53cf-4f33-9c2d-a474b586f24c None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:22.880544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ba28f-4797-41cd-957e-6e8241eb4d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:22.881327 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ba28f-4797-41cd-957e-6e8241eb4d40 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:22.881605 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ba28f-4797-41cd-957e-6e8241eb4d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:22.881877 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ba28f-4797-41cd-957e-6e8241eb4d40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:22.889129 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ba28f-4797-41cd-957e-6e8241eb4d40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:22.890590 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ba28f-4797-41cd-957e-6e8241eb4d40 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:22.903983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-586ba28f-4797-41cd-957e-6e8241eb4d40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:30:22.903983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:22.903983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:22.903983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:22.903983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:22.903983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:22.903983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:22.903983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:22.903983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:22.903983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:22.903983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:22.903983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:22.903983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:30:22.903983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:30:22.903983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:30:22.905891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:30:22.905891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:22.905891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:22.905891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:22.905891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:22.905891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:22.905891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:22.905891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:22.905891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:22.905891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:22.905891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:22.905891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:22.905891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:22.905891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:22.905891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:22.905891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:22.905891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:22.906883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:30:22.906883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:30:22.906883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:30:22.906883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:30:22.906883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:30:22.906883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:30:22.906883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:30:22.906883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:22.906883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 637/1244] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:30:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:22.911020 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692583b6-b4f5-4471-8195-928f4e8739ac None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:23.011283 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692583b6-b4f5-4471-8195-928f4e8739ac None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:23.043953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14770b6c-114e-4503-a76c-298f12719344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:23.044600 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14770b6c-114e-4503-a76c-298f12719344 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:23.044884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14770b6c-114e-4503-a76c-298f12719344 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:23.045161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14770b6c-114e-4503-a76c-298f12719344 None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:23.045939 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14770b6c-114e-4503-a76c-298f12719344 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:23.046247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14770b6c-114e-4503-a76c-298f12719344 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:23.047723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14770b6c-114e-4503-a76c-298f12719344 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:23.061252 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 608/1245] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:30:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:30:23.067345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ba28f-4797-41cd-957e-6e8241eb4d40 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:23.166785 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ba28f-4797-41cd-957e-6e8241eb4d40 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:23.200546 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067d57d-1753-4407-b14c-e672076709c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:23.201193 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067d57d-1753-4407-b14c-e672076709c4 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:23.201476 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067d57d-1753-4407-b14c-e672076709c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:23.201754 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067d57d-1753-4407-b14c-e672076709c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:23.202722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067d57d-1753-4407-b14c-e672076709c4 None admin] RBAC: Adding query filter params (role.id=c4bc6e72b9d647c5b2485d52d55f7dd4, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=3cd6f0a1286243e4add692db01028205) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:23.203032 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067d57d-1753-4407-b14c-e672076709c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:23.204459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067d57d-1753-4407-b14c-e672076709c4 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:23.210277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 638/1246] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:30:23 2022] GET /identity/v3/role_assignments?user.id=3cd6f0a1286243e4add692db01028205&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=c4bc6e72b9d647c5b2485d52d55f7dd4 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:25.293960 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96417c8-5678-418c-84eb-b6da22e3eebb None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:25.294523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96417c8-5678-418c-84eb-b6da22e3eebb None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:25.294523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96417c8-5678-418c-84eb-b6da22e3eebb None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:25.296047 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 609/1247] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:30:25.301261 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a31253-f77e-4e96-8d80-0fc077b74f4a None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:25.301702 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a31253-f77e-4e96-8d80-0fc077b74f4a None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:25.301843 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a31253-f77e-4e96-8d80-0fc077b74f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:25.306879 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:25.318547 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-c0a31253-f77e-4e96-8d80-0fc077b74f4a None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:30:25.319722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c0a31253-f77e-4e96-8d80-0fc077b74f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:30:25.385851 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 639/1248] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:30:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:30:25.410642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96417c8-5678-418c-84eb-b6da22e3eebb None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:25.429824 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:25.519397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96417c8-5678-418c-84eb-b6da22e3eebb None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:25.553720 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbff8af-0445-4541-9a9f-60d79c5b7dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:25.554557 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbff8af-0445-4541-9a9f-60d79c5b7dca None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:25.554839 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbff8af-0445-4541-9a9f-60d79c5b7dca None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:25.555110 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbff8af-0445-4541-9a9f-60d79c5b7dca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:25.565067 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbff8af-0445-4541-9a9f-60d79c5b7dca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:30:25.573272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-bdbff8af-0445-4541-9a9f-60d79c5b7dca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:30:25.573272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:25.573272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:25.573272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:25.573272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:25.573272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:25.573272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:25.573272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:25.573272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:25.573272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:25.573272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:25.573272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:25.573272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:30:25.573272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:30:25.573272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:30:25.573272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:30:25.581007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:30:25.581007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:30:25.581007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:30:25.581007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:30:25.581007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:25.581007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:25.581007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:25.581007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:25.581007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:25.581007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:25.581007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:25.581007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:25.581007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:25.581007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:25.581007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:25.581007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:25.582067 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:25.582067 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:25.582067 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:25.582067 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:25.582067 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:30:25.582067 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:30:25.582067 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:30:25.582067 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:30:25.582067 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:30:25.582067 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:30:25.582067 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:30:25.582067 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:30:25.582067 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 610/1249] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:30:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:25.582907 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a31253-f77e-4e96-8d80-0fc077b74f4a None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:25.682607 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a31253-f77e-4e96-8d80-0fc077b74f4a None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:25.712106 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:25.716908 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102591a1-8ed2-4bf8-bae7-259d1dda0d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:25.717560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102591a1-8ed2-4bf8-bae7-259d1dda0d59 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:25.717840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102591a1-8ed2-4bf8-bae7-259d1dda0d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:25.718112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102591a1-8ed2-4bf8-bae7-259d1dda0d59 None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:25.718957 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102591a1-8ed2-4bf8-bae7-259d1dda0d59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:25.719260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102591a1-8ed2-4bf8-bae7-259d1dda0d59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:25.720623 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102591a1-8ed2-4bf8-bae7-259d1dda0d59 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:25.728750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 640/1250] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:30:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:25.734524 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbff8af-0445-4541-9a9f-60d79c5b7dca None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:25.813044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:25.836566 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbff8af-0445-4541-9a9f-60d79c5b7dca None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:25.869889 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd65fb-ec26-433d-82ba-17546c90790a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:25.870689 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd65fb-ec26-433d-82ba-17546c90790a None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:25.870970 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd65fb-ec26-433d-82ba-17546c90790a None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:25.871241 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd65fb-ec26-433d-82ba-17546c90790a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:25.882708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd65fb-ec26-433d-82ba-17546c90790a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:25.884235 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd65fb-ec26-433d-82ba-17546c90790a None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:25.900886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-f4fd65fb-ec26-433d-82ba-17546c90790a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:25.900886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:25.900886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:25.900886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:25.900886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:25.900886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:25.900886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:25.900886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:25.900886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:25.900886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:25.900886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:25.900886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:25.900886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:30:25.900886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:30:25.900886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:30:25.901961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:30:25.901961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:25.901961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:25.901961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:25.901961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:25.901961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:25.901961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:25.901961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:25.901961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:25.901961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:25.901961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:25.901961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:25.901961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:25.901961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:25.901961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:25.901961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:25.901961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:25.903312 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:30:25.903312 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:30:25.903312 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:25.903312 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:30:25.903312 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 611/1251] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:30:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:25.907878 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102591a1-8ed2-4bf8-bae7-259d1dda0d59 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:26.006463 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102591a1-8ed2-4bf8-bae7-259d1dda0d59 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:26.039848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa9b54-1041-4929-88ce-f74e976d305a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:26.040504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa9b54-1041-4929-88ce-f74e976d305a None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:26.040787 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa9b54-1041-4929-88ce-f74e976d305a None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:26.041060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa9b54-1041-4929-88ce-f74e976d305a None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:26.041780 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa9b54-1041-4929-88ce-f74e976d305a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:26.042086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa9b54-1041-4929-88ce-f74e976d305a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:26.043480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa9b54-1041-4929-88ce-f74e976d305a None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:26.057705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 641/1252] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:30:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:26.063650 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd65fb-ec26-433d-82ba-17546c90790a None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:26.164203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd65fb-ec26-433d-82ba-17546c90790a None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:26.189713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:26.197375 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02328c70-c07e-459e-ac1f-f2baa4fce660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:26.198183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02328c70-c07e-459e-ac1f-f2baa4fce660 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:26.198523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02328c70-c07e-459e-ac1f-f2baa4fce660 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:26.198801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02328c70-c07e-459e-ac1f-f2baa4fce660 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:26.205944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02328c70-c07e-459e-ac1f-f2baa4fce660 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:26.207469 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02328c70-c07e-459e-ac1f-f2baa4fce660 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:26.221728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-02328c70-c07e-459e-ac1f-f2baa4fce660 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 18:30:26.221728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:26.221728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:26.221728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:26.221728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:26.221728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:26.221728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:26.221728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:26.221728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:26.221728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:26.221728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:26.221728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:26.221728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:30:26.221728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:30:26.221728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:30:26.221728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:30:26.222696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:26.222696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:26.222696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:30:26.222696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:30:26.222696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:30:26.222696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:30:26.222696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:26.222696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:26.222696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:26.222696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:26.222696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:26.222696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:26.222696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:26.222696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:26.222696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:26.222696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:26.222696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:26.222696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:26.223641 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:26.223641 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:26.223641 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:30:26.223641 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:30:26.223641 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:30:26.223641 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:30:26.223641 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:30:26.223641 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:30:26.223641 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 18:30:26.223641 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:30:26.223641 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 612/1253] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:30:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:26.229596 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa9b54-1041-4929-88ce-f74e976d305a None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:26.285018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:26.328917 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa9b54-1041-4929-88ce-f74e976d305a None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:26.362100 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad7642-4a96-4029-9d38-0cdc3eb7f15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:26.362769 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad7642-4a96-4029-9d38-0cdc3eb7f15f None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:26.363053 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad7642-4a96-4029-9d38-0cdc3eb7f15f None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:26.363323 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad7642-4a96-4029-9d38-0cdc3eb7f15f None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:26.364160 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad7642-4a96-4029-9d38-0cdc3eb7f15f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:26.364673 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad7642-4a96-4029-9d38-0cdc3eb7f15f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:26.367742 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad7642-4a96-4029-9d38-0cdc3eb7f15f None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:26.385367 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 642/1254] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Jun 27 18:30:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:26.391156 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02328c70-c07e-459e-ac1f-f2baa4fce660 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:26.493568 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02328c70-c07e-459e-ac1f-f2baa4fce660 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:26.526838 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1918a-09f2-41cd-9c84-a816fe16ddd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:26.527464 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1918a-09f2-41cd-9c84-a816fe16ddd7 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:26.527741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1918a-09f2-41cd-9c84-a816fe16ddd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:26.528011 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1918a-09f2-41cd-9c84-a816fe16ddd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:26.581380 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:26.593981 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1918a-09f2-41cd-9c84-a816fe16ddd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:26.595444 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1918a-09f2-41cd-9c84-a816fe16ddd7 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:26.694269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 613/1255] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:30:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 18:30:26.700223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad7642-4a96-4029-9d38-0cdc3eb7f15f None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:26.801664 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad7642-4a96-4029-9d38-0cdc3eb7f15f None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:26.814739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:26.835910 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61b32e-7edd-4527-947b-5b073a792fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:26.836621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61b32e-7edd-4527-947b-5b073a792fee None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:26.836940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61b32e-7edd-4527-947b-5b073a792fee None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:26.837327 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61b32e-7edd-4527-947b-5b073a792fee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:26.847269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d61b32e-7edd-4527-947b-5b073a792fee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:26.848931 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d61b32e-7edd-4527-947b-5b073a792fee None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:26.867298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-8d61b32e-7edd-4527-947b-5b073a792fee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:26.867298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:26.867298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:26.867298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:26.867298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:26.867298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:26.867298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:26.867298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:26.867298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:26.867298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:26.867298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:26.867298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:26.867298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:30:26.867298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:30:26.867298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:30:26.868380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:30:26.868380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:26.868380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:26.868380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:26.868380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:26.868380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:26.868380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:26.868380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:26.868380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:26.868380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:26.868380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:26.868380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:26.868380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:26.868380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:26.868380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:26.868380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:26.868380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:26.869378 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:30:26.869378 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:30:26.869378 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:26.869378 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:26.869378 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 643/1256] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:30:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:26.874198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1918a-09f2-41cd-9c84-a816fe16ddd7 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:26.961501 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:26.972997 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1918a-09f2-41cd-9c84-a816fe16ddd7 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:27.005458 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d6162-b0bf-475f-b325-8213c22e9d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:27.006111 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d6162-b0bf-475f-b325-8213c22e9d1f None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:27.006447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d6162-b0bf-475f-b325-8213c22e9d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:27.007242 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d6162-b0bf-475f-b325-8213c22e9d1f None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:27.007952 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d6162-b0bf-475f-b325-8213c22e9d1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:27.008249 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d6162-b0bf-475f-b325-8213c22e9d1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:27.009593 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d6162-b0bf-475f-b325-8213c22e9d1f None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:27.020317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 614/1257] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:27.026687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61b32e-7edd-4527-947b-5b073a792fee None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:27.122932 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61b32e-7edd-4527-947b-5b073a792fee None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:27.156727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677eaf3-2a23-49ac-84c1-fef8a6b3af28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:27.157499 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677eaf3-2a23-49ac-84c1-fef8a6b3af28 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:27.157782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677eaf3-2a23-49ac-84c1-fef8a6b3af28 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:27.158058 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677eaf3-2a23-49ac-84c1-fef8a6b3af28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:27.164902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677eaf3-2a23-49ac-84c1-fef8a6b3af28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:27.166336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677eaf3-2a23-49ac-84c1-fef8a6b3af28 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:27.179920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-f677eaf3-2a23-49ac-84c1-fef8a6b3af28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:30:27.179920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:27.179920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:27.179920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:27.179920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:27.179920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:27.179920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:27.179920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:27.179920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:27.179920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:27.179920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:27.179920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:27.179920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:30:27.179920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:30:27.179920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:30:27.181008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:30:27.181008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:27.181008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:27.181008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:27.181008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:27.181008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:27.181008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:27.181008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:27.181008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:27.181008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:27.181008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:27.181008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:27.181008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:27.181008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:27.181008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:27.181008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:27.181008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:27.182057 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:30:27.182057 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:30:27.182057 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:30:27.182057 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:30:27.182057 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:30:27.182057 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:30:27.182057 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:30:27.182057 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:27.182057 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 644/1258] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:30:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:27.187016 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d6162-b0bf-475f-b325-8213c22e9d1f None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:27.284964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d6162-b0bf-475f-b325-8213c22e9d1f None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:27.318027 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddbfe46-0b76-4aeb-b00a-4169c31e1725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:27.318705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbfe46-0b76-4aeb-b00a-4169c31e1725 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:27.322495 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbfe46-0b76-4aeb-b00a-4169c31e1725 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:27.322495 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbfe46-0b76-4aeb-b00a-4169c31e1725 None admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:27.322495 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddbfe46-0b76-4aeb-b00a-4169c31e1725 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:27.322495 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddbfe46-0b76-4aeb-b00a-4169c31e1725 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:27.322495 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddbfe46-0b76-4aeb-b00a-4169c31e1725 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:27.334237 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 615/1259] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:30:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:27.340430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677eaf3-2a23-49ac-84c1-fef8a6b3af28 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:27.366806 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:27.441643 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677eaf3-2a23-49ac-84c1-fef8a6b3af28 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:27.475578 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805dab8-c893-4a1a-b680-3f89cf76ca6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:27.476207 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805dab8-c893-4a1a-b680-3f89cf76ca6d None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:27.476488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805dab8-c893-4a1a-b680-3f89cf76ca6d None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:27.476755 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805dab8-c893-4a1a-b680-3f89cf76ca6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:27.477831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805dab8-c893-4a1a-b680-3f89cf76ca6d None admin] RBAC: Adding query filter params (role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=3cd6f0a1286243e4add692db01028205) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:27.478164 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805dab8-c893-4a1a-b680-3f89cf76ca6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:27.479599 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805dab8-c893-4a1a-b680-3f89cf76ca6d None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:27.485082 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 645/1260] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:30:27 2022] GET /identity/v3/role_assignments?user.id=3cd6f0a1286243e4add692db01028205&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:29.508179 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d435cd-de3f-4f3e-be71-7d6fba87dddf None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:29.508770 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d435cd-de3f-4f3e-be71-7d6fba87dddf None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:29.508770 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d435cd-de3f-4f3e-be71-7d6fba87dddf None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:29.510198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 616/1261] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:30:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:30:29.515517 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47403eb-0f25-4f11-87c7-8f88e7145ce8 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:29.515909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47403eb-0f25-4f11-87c7-8f88e7145ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:29.516080 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47403eb-0f25-4f11-87c7-8f88e7145ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:29.533061 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-b47403eb-0f25-4f11-87c7-8f88e7145ce8 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:30:29.534267 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-b47403eb-0f25-4f11-87c7-8f88e7145ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:30:29.600644 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 646/1262] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:30:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 18:30:29.623924 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d435cd-de3f-4f3e-be71-7d6fba87dddf None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:29.636049 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:29.724247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d435cd-de3f-4f3e-be71-7d6fba87dddf None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:29.759879 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914bd9e-3f2a-423c-ba8c-fe13152cd0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:29.760601 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914bd9e-3f2a-423c-ba8c-fe13152cd0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:29.760883 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914bd9e-3f2a-423c-ba8c-fe13152cd0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:29.761163 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914bd9e-3f2a-423c-ba8c-fe13152cd0e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:29.771278 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914bd9e-3f2a-423c-ba8c-fe13152cd0e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:30:29.779465 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-1914bd9e-3f2a-423c-ba8c-fe13152cd0e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:30:29.779465 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:29.779465 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:29.779465 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:29.779465 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:29.779465 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:29.779465 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:29.779465 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:29.779465 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:29.779465 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:29.779465 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:29.779465 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:29.779465 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:30:29.779465 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:30:29.779465 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:30:29.779465 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:30:29.785119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:30:29.785119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:30:29.785119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:30:29.785119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:30:29.785119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:29.785119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:29.785119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:29.785119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:29.785119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:29.785119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:29.785119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:29.785119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:29.785119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:29.785119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:29.785119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:29.785119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:29.786152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:29.786152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:29.786152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:29.786152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:29.786152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:30:29.786152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:30:29.786152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:30:29.786152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:30:29.786152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:30:29.786152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:30:29.786152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:30:29.786152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:30:29.786152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 617/1263] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:30:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:29.787041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47403eb-0f25-4f11-87c7-8f88e7145ce8 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:29.805842 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:29.888075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47403eb-0f25-4f11-87c7-8f88e7145ce8 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:29.922977 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a860f-fe07-4731-97a9-c29c64ec9aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:29.923636 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a860f-fe07-4731-97a9-c29c64ec9aec None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:29.923918 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a860f-fe07-4731-97a9-c29c64ec9aec None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:29.924224 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a860f-fe07-4731-97a9-c29c64ec9aec None admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:29.924970 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a860f-fe07-4731-97a9-c29c64ec9aec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:29.925275 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a860f-fe07-4731-97a9-c29c64ec9aec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:29.926751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a860f-fe07-4731-97a9-c29c64ec9aec None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:29.935068 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 647/1264] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:30:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:30:29.940719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914bd9e-3f2a-423c-ba8c-fe13152cd0e6 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:30.014424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:30.047576 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914bd9e-3f2a-423c-ba8c-fe13152cd0e6 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:30.079904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf225279-57ed-4c84-89cc-7284cf2ae2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:30.080678 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf225279-57ed-4c84-89cc-7284cf2ae2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:30.080962 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf225279-57ed-4c84-89cc-7284cf2ae2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:30.081236 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf225279-57ed-4c84-89cc-7284cf2ae2c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:30.092517 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf225279-57ed-4c84-89cc-7284cf2ae2c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:30.093952 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf225279-57ed-4c84-89cc-7284cf2ae2c5 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:30.110999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-bf225279-57ed-4c84-89cc-7284cf2ae2c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:30.110999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:30.110999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:30.110999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:30.110999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:30.110999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:30.110999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:30.110999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:30.110999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:30.110999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:30.110999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:30.110999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:30.110999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:30:30.110999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:30:30.110999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:30:30.112054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:30:30.112054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:30.112054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:30.112054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:30.112054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:30.112054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:30.112054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:30.112054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:30.112054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:30.112054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:30.112054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:30.112054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:30.112054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:30.112054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:30.112054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:30.112054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:30.112054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:30.113431 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:30:30.113431 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:30:30.113431 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:30.113431 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:30:30.113431 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 618/1265] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:30:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:30.118326 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a860f-fe07-4731-97a9-c29c64ec9aec None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:30.200970 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:30.219600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a860f-fe07-4731-97a9-c29c64ec9aec None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:30.252728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c47791-4a67-40f4-b260-cb460c200868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:30.253376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c47791-4a67-40f4-b260-cb460c200868 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:30.253660 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c47791-4a67-40f4-b260-cb460c200868 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:30.253936 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c47791-4a67-40f4-b260-cb460c200868 None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:30.254683 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c47791-4a67-40f4-b260-cb460c200868 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:30.255120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c47791-4a67-40f4-b260-cb460c200868 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:30.256410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c47791-4a67-40f4-b260-cb460c200868 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:30.269057 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 648/1266] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:30.274555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf225279-57ed-4c84-89cc-7284cf2ae2c5 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:30.379553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf225279-57ed-4c84-89cc-7284cf2ae2c5 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:30.400207 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:30.415850 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adef596-dfb8-45e1-be57-4a88e4ee70ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:30.416669 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adef596-dfb8-45e1-be57-4a88e4ee70ab None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:30.416949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adef596-dfb8-45e1-be57-4a88e4ee70ab None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:30.417223 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adef596-dfb8-45e1-be57-4a88e4ee70ab None admin] PATH_INFO: `/v3/users/placement` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:30.424568 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adef596-dfb8-45e1-be57-4a88e4ee70ab None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:30.426033 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adef596-dfb8-45e1-be57-4a88e4ee70ab None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:30.447118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-5adef596-dfb8-45e1-be57-4a88e4ee70ab None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 18:30:30.447118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:30.447118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:30.447118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:30.447118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:30.447118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:30.447118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:30.447118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:30.447118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:30.447118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:30.447118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:30.447118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:30.447118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:30:30.447118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:30:30.447118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:30:30.447118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:30:30.448506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:30.448506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:30.448506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:30:30.448506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:30:30.448506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:30:30.448506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:30:30.448506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:30.448506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:30.448506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:30.448506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:30.448506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:30.448506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:30.448506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:30.448506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:30.448506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:30.448506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:30.448506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:30.448506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:30.449712 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:30.449712 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:30.449712 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:30:30.449712 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:30:30.449712 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:30:30.449712 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:30:30.449712 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:30:30.449712 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:30:30.449712 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 18:30:30.449712 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:30:30.449712 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 619/1267] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:30:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:30.454216 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c47791-4a67-40f4-b260-cb460c200868 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:30.556990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c47791-4a67-40f4-b260-cb460c200868 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:30.595898 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e643790-c8f4-4ca2-83ae-63aae109ff94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:30.596571 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e643790-c8f4-4ca2-83ae-63aae109ff94 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:30.596878 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e643790-c8f4-4ca2-83ae-63aae109ff94 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:30.597156 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e643790-c8f4-4ca2-83ae-63aae109ff94 None admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:30.598291 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e643790-c8f4-4ca2-83ae-63aae109ff94 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:30.598848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e643790-c8f4-4ca2-83ae-63aae109ff94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:30.600322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e643790-c8f4-4ca2-83ae-63aae109ff94 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:30.613029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:30.624952 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 649/1268] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Jun 27 18:30:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:30.630881 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adef596-dfb8-45e1-be57-4a88e4ee70ab None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:30.758023 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adef596-dfb8-45e1-be57-4a88e4ee70ab None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:30.809326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47968257-045b-40d4-80aa-c50f1d2d71d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:30.810482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47968257-045b-40d4-80aa-c50f1d2d71d3 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:30.810898 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47968257-045b-40d4-80aa-c50f1d2d71d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:30.811309 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47968257-045b-40d4-80aa-c50f1d2d71d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:30.826207 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47968257-045b-40d4-80aa-c50f1d2d71d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:30.828447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47968257-045b-40d4-80aa-c50f1d2d71d3 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:30.855726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-47968257-045b-40d4-80aa-c50f1d2d71d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:30.855726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:30.855726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:30.855726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:30.855726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:30.855726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:30.855726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:30.855726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:30.855726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:30.855726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:30.855726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:30.855726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:30.855726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:30:30.855726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:30:30.855726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:30:30.856888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:30:30.856888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:30.856888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:30.856888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:30.856888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:30.856888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:30.856888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:30.856888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:30.856888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:30.856888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:30.856888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:30.856888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:30.856888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:30.856888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:30.856888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:30.856888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:30.856888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:30.857940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:30:30.857940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:30:30.857940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:30.857940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:30:30.857940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 620/1269] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:30:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:30.863822 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e643790-c8f4-4ca2-83ae-63aae109ff94 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:30.978127 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e643790-c8f4-4ca2-83ae-63aae109ff94 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:31.012716 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b64630-50af-4296-857b-7ee4d753276e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:31.013368 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b64630-50af-4296-857b-7ee4d753276e None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:31.013648 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b64630-50af-4296-857b-7ee4d753276e None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:31.013917 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b64630-50af-4296-857b-7ee4d753276e None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:31.014664 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b64630-50af-4296-857b-7ee4d753276e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:31.014964 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b64630-50af-4296-857b-7ee4d753276e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:31.016323 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b64630-50af-4296-857b-7ee4d753276e None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:31.028208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 650/1270] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:31.033972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47968257-045b-40d4-80aa-c50f1d2d71d3 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:31.065681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:31.133706 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47968257-045b-40d4-80aa-c50f1d2d71d3 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:31.172702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b0cd7-dcea-488d-a080-b7a7d63236ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:31.173486 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b0cd7-dcea-488d-a080-b7a7d63236ec None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:31.173770 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b0cd7-dcea-488d-a080-b7a7d63236ec None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:31.174047 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b0cd7-dcea-488d-a080-b7a7d63236ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:31.180983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b0cd7-dcea-488d-a080-b7a7d63236ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:31.182510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b0cd7-dcea-488d-a080-b7a7d63236ec None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:31.196095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-460b0cd7-dcea-488d-a080-b7a7d63236ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:30:31.196095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:31.196095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:31.196095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:31.196095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:31.196095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:31.196095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:31.196095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:31.196095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:31.196095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:31.196095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:31.196095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:31.196095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:30:31.196095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:30:31.196095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:30:31.197164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:30:31.197164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:31.197164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:31.197164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:31.197164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:31.197164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:31.197164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:31.197164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:31.197164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:31.197164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:31.197164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:31.197164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:31.197164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:31.197164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:31.197164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:31.197164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:31.197164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:31.198227 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:30:31.198227 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:30:31.198227 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:30:31.198227 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:30:31.198227 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:30:31.198227 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:30:31.198227 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:30:31.198227 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:30:31.198227 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 621/1271] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:30:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:31.203156 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b64630-50af-4296-857b-7ee4d753276e None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:31.262343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:31.303903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b64630-50af-4296-857b-7ee4d753276e None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:31.337508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a3139-bfa4-4053-8728-28da1988b9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:31.338163 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a3139-bfa4-4053-8728-28da1988b9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:31.338474 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a3139-bfa4-4053-8728-28da1988b9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:31.338751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a3139-bfa4-4053-8728-28da1988b9f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:31.339542 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a3139-bfa4-4053-8728-28da1988b9f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:31.339851 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a3139-bfa4-4053-8728-28da1988b9f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:31.341323 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a3139-bfa4-4053-8728-28da1988b9f9 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:31.355993 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 651/1272] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:30:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:31.362009 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b0cd7-dcea-488d-a080-b7a7d63236ec None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:31.461459 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:31.474890 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b0cd7-dcea-488d-a080-b7a7d63236ec None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:31.520956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e7c085-879c-4d49-9bdb-bde78b34eb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:31.522754 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7c085-879c-4d49-9bdb-bde78b34eb3b None admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:31.523171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7c085-879c-4d49-9bdb-bde78b34eb3b None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:31.523558 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7c085-879c-4d49-9bdb-bde78b34eb3b None admin] PATH_INFO: `/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/3cd6f0a1286243e4add692db01028205/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:31.526457 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e7c085-879c-4d49-9bdb-bde78b34eb3b None admin] RBAC: Authorizing `identity:create_grant(project_id=84ecf552f8364884b4aeff3428cb81e4, user_id=3cd6f0a1286243e4add692db01028205, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:31.528683 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e7c085-879c-4d49-9bdb-bde78b34eb3b None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:31.731488 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 622/1273] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Jun 27 18:30:31 2022] PUT /identity/v3/projects/84ecf552f8364884b4aeff3428cb81e4/users/3cd6f0a1286243e4add692db01028205/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:30:33.766572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4c201-7c5c-4f27-9750-154a36caea70 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:33.767082 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4c201-7c5c-4f27-9750-154a36caea70 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:33.767176 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4c201-7c5c-4f27-9750-154a36caea70 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:33.768474 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 652/1274] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:30:33.774011 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172cd650-f133-4bc8-bad1-ec0cf01d7585 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:33.774484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172cd650-f133-4bc8-bad1-ec0cf01d7585 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:33.774586 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172cd650-f133-4bc8-bad1-ec0cf01d7585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:33.791782 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-172cd650-f133-4bc8-bad1-ec0cf01d7585 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:30:33.792944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-172cd650-f133-4bc8-bad1-ec0cf01d7585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:30:33.860099 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 623/1275] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:30:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 18:30:33.884516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4c201-7c5c-4f27-9750-154a36caea70 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:33.984779 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4c201-7c5c-4f27-9750-154a36caea70 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:34.006822 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:34.019325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4522e32-1ee8-4858-9e33-721777c979bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:34.020128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4522e32-1ee8-4858-9e33-721777c979bd None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:34.020409 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4522e32-1ee8-4858-9e33-721777c979bd None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:34.020683 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4522e32-1ee8-4858-9e33-721777c979bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:34.031570 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4522e32-1ee8-4858-9e33-721777c979bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 18:30:34.039613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-b4522e32-1ee8-4858-9e33-721777c979bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:30:34.039613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:34.039613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:34.039613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:34.039613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:34.039613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:34.039613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:34.039613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:34.039613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:34.039613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:34.039613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:34.039613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:34.039613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 18:30:34.039613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 18:30:34.039613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 18:30:34.039613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 18:30:34.040777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 18:30:34.040777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 18:30:34.040777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 18:30:34.040777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 18:30:34.040777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:34.040777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:34.040777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:34.040777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:34.040777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:34.040777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:34.040777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:34.040777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:34.040777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:34.040777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:34.040777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:34.040777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:34.042016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:34.042016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:34.042016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:34.042016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:34.042016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 18:30:34.042016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 18:30:34.042016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 18:30:34.042016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 18:30:34.042016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 18:30:34.042016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 18:30:34.042016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 18:30:34.042016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:34.042016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 653/1276] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Jun 27 18:30:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:34.046794 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172cd650-f133-4bc8-bad1-ec0cf01d7585 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:34.127020 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:34.145028 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172cd650-f133-4bc8-bad1-ec0cf01d7585 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:34.177846 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee890c-de63-43de-8ed2-9b5cf929d78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:34.179240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee890c-de63-43de-8ed2-9b5cf929d78b None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:34.179512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee890c-de63-43de-8ed2-9b5cf929d78b None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:34.179789 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee890c-de63-43de-8ed2-9b5cf929d78b None admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:34.180527 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee890c-de63-43de-8ed2-9b5cf929d78b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:34.180832 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee890c-de63-43de-8ed2-9b5cf929d78b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:34.182181 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee890c-de63-43de-8ed2-9b5cf929d78b None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:34.189947 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 624/1277] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Jun 27 18:30:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:34.196474 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4522e32-1ee8-4858-9e33-721777c979bd None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:34.295850 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4522e32-1ee8-4858-9e33-721777c979bd None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:34.329191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fe362-d246-4338-a79d-3247606183f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:34.329968 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fe362-d246-4338-a79d-3247606183f9 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:34.330256 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fe362-d246-4338-a79d-3247606183f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:34.330631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fe362-d246-4338-a79d-3247606183f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:34.340959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5fe362-d246-4338-a79d-3247606183f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:34.343664 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5fe362-d246-4338-a79d-3247606183f9 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:34.359376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-3e5fe362-d246-4338-a79d-3247606183f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:34.359376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:34.359376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:34.359376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:34.359376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:34.359376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:34.359376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:34.359376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:34.359376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:34.359376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:34.359376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:34.359376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:34.359376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:30:34.359376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:30:34.359376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:30:34.360299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:30:34.360299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:34.360299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:34.360299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:34.360299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:34.360299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:34.360299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:34.360299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:34.360299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:34.360299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:34.360299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:34.360299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:34.360299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:34.360299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:34.360299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:34.360299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:34.360299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:34.361301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:30:34.361301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:30:34.361301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:34.361301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:34.361301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 654/1278] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:30:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:34.367044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee890c-de63-43de-8ed2-9b5cf929d78b None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:34.465247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee890c-de63-43de-8ed2-9b5cf929d78b None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:34.501256 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc70a68-da7f-46e6-b81a-84dfd7a7df27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:34.501891 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc70a68-da7f-46e6-b81a-84dfd7a7df27 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:34.502173 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc70a68-da7f-46e6-b81a-84dfd7a7df27 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:34.502480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc70a68-da7f-46e6-b81a-84dfd7a7df27 None admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:34.503196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc70a68-da7f-46e6-b81a-84dfd7a7df27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:34.503497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc70a68-da7f-46e6-b81a-84dfd7a7df27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:34.504846 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc70a68-da7f-46e6-b81a-84dfd7a7df27 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:34.516271 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 625/1279] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:34.522501 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fe362-d246-4338-a79d-3247606183f9 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:34.553170 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:34.622671 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fe362-d246-4338-a79d-3247606183f9 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:34.656338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aca372-2573-4478-9e28-9d406e924efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:34.657128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aca372-2573-4478-9e28-9d406e924efd None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:34.657410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aca372-2573-4478-9e28-9d406e924efd None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:34.657682 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aca372-2573-4478-9e28-9d406e924efd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:34.665202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aca372-2573-4478-9e28-9d406e924efd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:34.666773 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aca372-2573-4478-9e28-9d406e924efd None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:34.680963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-a4aca372-2573-4478-9e28-9d406e924efd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 18:30:34.680963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:34.680963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:34.680963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:34.680963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:34.680963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:34.680963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:34.680963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:34.680963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:34.680963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:34.680963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:34.680963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:34.680963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 18:30:34.680963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 18:30:34.680963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 18:30:34.680963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 18:30:34.682134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:34.682134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:34.682134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 18:30:34.682134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:30:34.682134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 18:30:34.682134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 18:30:34.682134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:34.682134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:34.682134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:34.682134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:34.682134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:34.682134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:34.682134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:34.682134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:34.682134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:34.682134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:34.682134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:34.682134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:34.683283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:34.683283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:34.683283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 18:30:34.683283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 18:30:34.683283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 18:30:34.683283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 18:30:34.683283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 18:30:34.683283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 18:30:34.683283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 18:30:34.683283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:34.683283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 655/1280] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:30:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:34.688018 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc70a68-da7f-46e6-b81a-84dfd7a7df27 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:34.691759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:34.816774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc70a68-da7f-46e6-b81a-84dfd7a7df27 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:34.855547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87554d9d-9e29-443e-b2fa-721a58ecf3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:34.856215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87554d9d-9e29-443e-b2fa-721a58ecf3bf None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:34.856504 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87554d9d-9e29-443e-b2fa-721a58ecf3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:34.856764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87554d9d-9e29-443e-b2fa-721a58ecf3bf None admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:34.857605 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87554d9d-9e29-443e-b2fa-721a58ecf3bf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:34.857914 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87554d9d-9e29-443e-b2fa-721a58ecf3bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:34.859371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87554d9d-9e29-443e-b2fa-721a58ecf3bf None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:34.879151 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 626/1281] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Jun 27 18:30:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:34.884962 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aca372-2573-4478-9e28-9d406e924efd None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:34.984388 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:34.999142 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aca372-2573-4478-9e28-9d406e924efd None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:35.030897 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e1f5f-1ee0-432e-9f5c-e42d9dc82e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:35.031553 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e1f5f-1ee0-432e-9f5c-e42d9dc82e99 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:35.033985 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e1f5f-1ee0-432e-9f5c-e42d9dc82e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:35.033985 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e1f5f-1ee0-432e-9f5c-e42d9dc82e99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:35.104976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e1f5f-1ee0-432e-9f5c-e42d9dc82e99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:35.106483 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0e1f5f-1ee0-432e-9f5c-e42d9dc82e99 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:35.204089 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:35.215300 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 656/1282] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Jun 27 18:30:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 18:30:35.221140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87554d9d-9e29-443e-b2fa-721a58ecf3bf None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:35.282626 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:35.320836 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87554d9d-9e29-443e-b2fa-721a58ecf3bf None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:35.353862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a116e0-7ec5-470a-b11b-93885329d447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:35.354728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a116e0-7ec5-470a-b11b-93885329d447 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:35.355060 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a116e0-7ec5-470a-b11b-93885329d447 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:35.355291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a116e0-7ec5-470a-b11b-93885329d447 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:35.365662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a116e0-7ec5-470a-b11b-93885329d447 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:35.367135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a116e0-7ec5-470a-b11b-93885329d447 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:35.385188 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-64a116e0-7ec5-470a-b11b-93885329d447 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:35.385188 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:35.385188 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:35.385188 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:35.385188 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:35.385188 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:35.385188 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:35.385188 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:35.385188 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:35.385188 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:35.385188 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:35.385188 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:35.385188 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 18:30:35.385188 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 18:30:35.385188 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 18:30:35.386343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 18:30:35.386343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:35.386343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:35.386343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:35.386343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:35.386343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:35.386343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:35.386343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:35.386343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:35.386343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:35.386343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:35.386343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:35.386343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:35.386343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:35.386343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:35.386343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:35.386343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:35.387470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 18:30:35.387470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 18:30:35.387470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 18:30:35.387470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:30:35.387470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 627/1283] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:30:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:35.393110 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e1f5f-1ee0-432e-9f5c-e42d9dc82e99 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:35.491736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0e1f5f-1ee0-432e-9f5c-e42d9dc82e99 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:35.525771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225e1b1-818c-4755-9d6c-63ac7b8e4a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:35.526489 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225e1b1-818c-4755-9d6c-63ac7b8e4a85 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:35.526770 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225e1b1-818c-4755-9d6c-63ac7b8e4a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:35.527042 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225e1b1-818c-4755-9d6c-63ac7b8e4a85 None admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:35.527786 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225e1b1-818c-4755-9d6c-63ac7b8e4a85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:35.528082 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225e1b1-818c-4755-9d6c-63ac7b8e4a85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:35.529426 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225e1b1-818c-4755-9d6c-63ac7b8e4a85 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:35.540766 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 657/1284] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Jun 27 18:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:35.546487 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a116e0-7ec5-470a-b11b-93885329d447 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:35.646336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a116e0-7ec5-470a-b11b-93885329d447 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:35.659571 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:35.679230 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1ff76-fb19-442b-b213-97eb4b176ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:35.679996 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1ff76-fb19-442b-b213-97eb4b176ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:35.680271 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1ff76-fb19-442b-b213-97eb4b176ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:35.680539 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1ff76-fb19-442b-b213-97eb4b176ae5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:35.686946 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e1ff76-fb19-442b-b213-97eb4b176ae5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:35.688365 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e1ff76-fb19-442b-b213-97eb4b176ae5 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:35.702871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application [None req-88e1ff76-fb19-442b-b213-97eb4b176ae5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:30:35.702871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:35.702871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:35.702871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:35.702871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:35.702871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:35.702871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:35.702871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:35.702871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:35.702871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:35.702871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:35.702871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:35.702871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 18:30:35.702871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 18:30:35.702871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 18:30:35.703933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 18:30:35.703933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:35.703933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:35.703933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:35.703933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:35.703933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:35.703933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:35.703933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:35.703933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:35.703933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:35.703933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:35.703933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:35.703933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:35.703933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:35.703933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:35.703933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:35.703933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:35.704925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 18:30:35.704925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 18:30:35.704925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 18:30:35.704925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 18:30:35.704925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 18:30:35.704925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 18:30:35.704925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 18:30:35.704925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: ERROR keystone.server.flask.application Jun 27 18:30:35.704925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 628/1285] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Jun 27 18:30:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:35.710181 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225e1b1-818c-4755-9d6c-63ac7b8e4a85 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:35.761420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:35.808612 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225e1b1-818c-4755-9d6c-63ac7b8e4a85 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:35.841677 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da224b-5865-4d2a-90d5-e89393c2d4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:35.842320 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da224b-5865-4d2a-90d5-e89393c2d4de None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:35.842695 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da224b-5865-4d2a-90d5-e89393c2d4de None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:35.842972 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da224b-5865-4d2a-90d5-e89393c2d4de None admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:35.843750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da224b-5865-4d2a-90d5-e89393c2d4de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:35.844056 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da224b-5865-4d2a-90d5-e89393c2d4de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:35.845475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da224b-5865-4d2a-90d5-e89393c2d4de None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:35.857488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 658/1286] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Jun 27 18:30:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:35.863470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1ff76-fb19-442b-b213-97eb4b176ae5 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:35.960222 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1ff76-fb19-442b-b213-97eb4b176ae5 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:35.993871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457825f-f1ae-4f5a-b92f-232d8c09514c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:35.994583 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457825f-f1ae-4f5a-b92f-232d8c09514c None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:35.994866 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457825f-f1ae-4f5a-b92f-232d8c09514c None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:35.995135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457825f-f1ae-4f5a-b92f-232d8c09514c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:35.995942 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0457825f-f1ae-4f5a-b92f-232d8c09514c None admin] RBAC: Adding query filter params (role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f, scope.project.id=84ecf552f8364884b4aeff3428cb81e4, user.id=3cd6f0a1286243e4add692db01028205) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:35.996246 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0457825f-f1ae-4f5a-b92f-232d8c09514c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:35.997647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0457825f-f1ae-4f5a-b92f-232d8c09514c None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:36.003431 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 629/1287] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Jun 27 18:30:35 2022] GET /identity/v3/role_assignments?user.id=3cd6f0a1286243e4add692db01028205&scope.project.id=84ecf552f8364884b4aeff3428cb81e4&role.id=ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:38.126954 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd640acf-9be2-47df-b18a-f40fe14651ad None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:38.127489 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd640acf-9be2-47df-b18a-f40fe14651ad None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:38.127489 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd640acf-9be2-47df-b18a-f40fe14651ad None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:38.128876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 659/1288] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:30:38.134657 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88de42-ba17-49d3-89fd-65dcf78b9ee8 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:38.135096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88de42-ba17-49d3-89fd-65dcf78b9ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:38.135200 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88de42-ba17-49d3-89fd-65dcf78b9ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:38.152331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-cc88de42-ba17-49d3-89fd-65dcf78b9ee8 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:30:38.153500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-cc88de42-ba17-49d3-89fd-65dcf78b9ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:30:38.181904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:38.218008 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 630/1289] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:30:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:30:38.241244 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd640acf-9be2-47df-b18a-f40fe14651ad None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:38.341872 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd640acf-9be2-47df-b18a-f40fe14651ad None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:38.352900 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:38.375039 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb10ce-079f-465a-b3df-027456d4f45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:38.375811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb10ce-079f-465a-b3df-027456d4f45a None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:38.376092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb10ce-079f-465a-b3df-027456d4f45a None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:38.376365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb10ce-079f-465a-b3df-027456d4f45a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:38.377054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb10ce-079f-465a-b3df-027456d4f45a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:38.378366 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb10ce-079f-465a-b3df-027456d4f45a None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:38.391066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-4bbb10ce-079f-465a-b3df-027456d4f45a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 18:30:38.391066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:38.391066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:30:38.391066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:30:38.391066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:30:38.391066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:30:38.391066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:30:38.391066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:30:38.391066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 18:30:38.391066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:38.391066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:30:38.391066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:38.391066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:38.391066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:38.391066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:38.391066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:38.391066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:38.392343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:38.392343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:38.392343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:38.392343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:38.392343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:38.392343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:38.392343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:30:38.392343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:30:38.392343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:30:38.392343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:30:38.392343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:38.392343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:38.392343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:38.392343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:38.392343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:38.392343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:38.393410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:38.393410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:38.393410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:38.393410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:38.393410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:38.393410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:38.393410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:38.393410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:38.393410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:38.393410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:38.393410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:30:38.393410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:30:38.393410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 18:30:38.393410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:38.393410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 660/1290] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Jun 27 18:30:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:38.399116 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88de42-ba17-49d3-89fd-65dcf78b9ee8 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:38.498372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88de42-ba17-49d3-89fd-65dcf78b9ee8 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:38.531670 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7734e0b-c305-4d3d-8bd1-1d0352e53bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:38.532346 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7734e0b-c305-4d3d-8bd1-1d0352e53bed None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:38.532625 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7734e0b-c305-4d3d-8bd1-1d0352e53bed None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:38.532896 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7734e0b-c305-4d3d-8bd1-1d0352e53bed None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:38.533599 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7734e0b-c305-4d3d-8bd1-1d0352e53bed None admin] RBAC: Adding query filter params (name=placement) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:38.533955 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7734e0b-c305-4d3d-8bd1-1d0352e53bed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:38.535440 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7734e0b-c305-4d3d-8bd1-1d0352e53bed None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:38.540252 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 631/1291] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Jun 27 18:30:38 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:38.545969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb10ce-079f-465a-b3df-027456d4f45a None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:38.646932 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb10ce-079f-465a-b3df-027456d4f45a None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:38.680244 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c285fdd-fd5f-4bc8-b9d6-d8029d6cb5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:38.680893 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c285fdd-fd5f-4bc8-b9d6-d8029d6cb5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:38.681174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c285fdd-fd5f-4bc8-b9d6-d8029d6cb5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:38.681448 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c285fdd-fd5f-4bc8-b9d6-d8029d6cb5c6 None admin] PATH_INFO: `/v3/services` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:38.682146 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c285fdd-fd5f-4bc8-b9d6-d8029d6cb5c6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:38.682475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c285fdd-fd5f-4bc8-b9d6-d8029d6cb5c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:38.684089 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c285fdd-fd5f-4bc8-b9d6-d8029d6cb5c6 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:38.688264 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 661/1292] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Jun 27 18:30:38 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:40.718670 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed815450-a651-4172-80c1-44811493ed69 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:40.719184 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed815450-a651-4172-80c1-44811493ed69 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:40.719278 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed815450-a651-4172-80c1-44811493ed69 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:40.720613 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 632/1293] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:30:40.725851 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe2509-9a4f-4416-9a61-bf9ae8294a42 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:40.726344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe2509-9a4f-4416-9a61-bf9ae8294a42 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:40.726471 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe2509-9a4f-4416-9a61-bf9ae8294a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:40.745993 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-cffe2509-9a4f-4416-9a61-bf9ae8294a42 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:30:40.747354 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-cffe2509-9a4f-4416-9a61-bf9ae8294a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:30:40.778519 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:40.816638 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 662/1294] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:30:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:30:40.840464 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed815450-a651-4172-80c1-44811493ed69 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:40.872802 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:40.942530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed815450-a651-4172-80c1-44811493ed69 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:40.978463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f95ab-8dc0-425d-850c-53419682e0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:40.979066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f95ab-8dc0-425d-850c-53419682e0ff None admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:40.979348 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f95ab-8dc0-425d-850c-53419682e0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:40.979615 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f95ab-8dc0-425d-850c-53419682e0ff None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:40.980315 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f95ab-8dc0-425d-850c-53419682e0ff None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:40.981662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f95ab-8dc0-425d-850c-53419682e0ff None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:41.099740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 633/1295] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Jun 27 18:30:40 2022] POST /identity/v3/services => generated 246 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:30:43.166786 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080747a-4199-408f-ace9-f30072f49504 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:43.167345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080747a-4199-408f-ace9-f30072f49504 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:43.167445 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080747a-4199-408f-ace9-f30072f49504 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:43.168795 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 663/1296] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:30:43.173981 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b164f-86d4-4dba-81d5-ef29ce1f4147 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:43.174447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b164f-86d4-4dba-81d5-ef29ce1f4147 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:43.174574 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b164f-86d4-4dba-81d5-ef29ce1f4147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:43.192167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-b78b164f-86d4-4dba-81d5-ef29ce1f4147 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:30:43.194190 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-b78b164f-86d4-4dba-81d5-ef29ce1f4147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:30:43.268007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 634/1297] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:30:43 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 18:30:43.291615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080747a-4199-408f-ace9-f30072f49504 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:43.393175 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080747a-4199-408f-ace9-f30072f49504 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:43.426874 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7921c55-bf63-4ac0-afae-a1e8aeb00580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:43.428208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7921c55-bf63-4ac0-afae-a1e8aeb00580 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:43.428494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7921c55-bf63-4ac0-afae-a1e8aeb00580 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:43.428766 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7921c55-bf63-4ac0-afae-a1e8aeb00580 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:43.429413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7921c55-bf63-4ac0-afae-a1e8aeb00580 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:43.430852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7921c55-bf63-4ac0-afae-a1e8aeb00580 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:43.442374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-b7921c55-bf63-4ac0-afae-a1e8aeb00580 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 18:30:43.442374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:43.442374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:30:43.442374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:30:43.442374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:30:43.442374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:30:43.442374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:30:43.442374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:30:43.442374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 18:30:43.442374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:43.442374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:30:43.442374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:43.442374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:43.442374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:43.442374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:43.442374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:43.442374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:43.444693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:43.444693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:43.444693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:43.444693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:43.444693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:43.444693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:43.444693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:30:43.444693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:30:43.444693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:30:43.444693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:30:43.444693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:43.444693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:43.444693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:43.444693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:43.444693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:43.444693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:43.445631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:43.445631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:43.445631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:43.445631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:43.445631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:43.445631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:43.445631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:43.445631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:43.445631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:43.445631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:43.445631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:30:43.445631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:30:43.445631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 18:30:43.445631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:43.445631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 664/1298] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Jun 27 18:30:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:43.449807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b164f-86d4-4dba-81d5-ef29ce1f4147 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:43.467771 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:43.550762 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b164f-86d4-4dba-81d5-ef29ce1f4147 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:43.585566 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a647f-1479-4eb2-ab9a-664715b88380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:43.586205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a647f-1479-4eb2-ab9a-664715b88380 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:43.586509 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a647f-1479-4eb2-ab9a-664715b88380 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:43.586819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a647f-1479-4eb2-ab9a-664715b88380 None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:43.587476 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a647f-1479-4eb2-ab9a-664715b88380 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:43.587782 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a647f-1479-4eb2-ab9a-664715b88380 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:43.589123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a647f-1479-4eb2-ab9a-664715b88380 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:43.594116 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 635/1299] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Jun 27 18:30:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:43.600523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7921c55-bf63-4ac0-afae-a1e8aeb00580 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:43.633280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:43.700932 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7921c55-bf63-4ac0-afae-a1e8aeb00580 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:43.734555 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ba18f2-5ce4-4d43-99fc-122d63df44b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:43.735211 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba18f2-5ce4-4d43-99fc-122d63df44b6 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:43.735495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba18f2-5ce4-4d43-99fc-122d63df44b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:43.735772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba18f2-5ce4-4d43-99fc-122d63df44b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:43.736561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ba18f2-5ce4-4d43-99fc-122d63df44b6 None admin] RBAC: Adding query filter params (interface=public, service_id=806017531c2e4374866bce1cb9d318bd, region_id=RegionOne) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:43.736872 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ba18f2-5ce4-4d43-99fc-122d63df44b6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:43.738266 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ba18f2-5ce4-4d43-99fc-122d63df44b6 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:43.744441 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 665/1300] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Jun 27 18:30:43 2022] GET /identity/v3/endpoints?service_id=806017531c2e4374866bce1cb9d318bd&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:43.750047 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a647f-1479-4eb2-ab9a-664715b88380 None admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:43.831315 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:43.849846 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a647f-1479-4eb2-ab9a-664715b88380 None admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:43.884332 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8833f3b-5ec3-4b77-af26-89fc930b07a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:43.884982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8833f3b-5ec3-4b77-af26-89fc930b07a7 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:43.885266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8833f3b-5ec3-4b77-af26-89fc930b07a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:43.885544 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8833f3b-5ec3-4b77-af26-89fc930b07a7 None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:43.886224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8833f3b-5ec3-4b77-af26-89fc930b07a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:43.887625 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8833f3b-5ec3-4b77-af26-89fc930b07a7 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:43.894294 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 636/1301] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Jun 27 18:30:43 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:30:45.924609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00f0b4-53d3-4887-ba23-11aeb35d2085 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:45.925186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00f0b4-53d3-4887-ba23-11aeb35d2085 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:45.925186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00f0b4-53d3-4887-ba23-11aeb35d2085 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:45.926632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 666/1302] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:30:45.932196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba00f81-3683-4abc-981c-7c3230e39aae None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:45.932572 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba00f81-3683-4abc-981c-7c3230e39aae None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:45.932572 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba00f81-3683-4abc-981c-7c3230e39aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:45.949771 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-8ba00f81-3683-4abc-981c-7c3230e39aae None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:30:45.950985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-8ba00f81-3683-4abc-981c-7c3230e39aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:30:46.019221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 637/1303] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:30:45 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 18:30:46.041842 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00f0b4-53d3-4887-ba23-11aeb35d2085 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:46.142823 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:46.149802 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00f0b4-53d3-4887-ba23-11aeb35d2085 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:46.184245 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57082cf1-6992-4d2f-b5d0-01714fdc9734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:46.185255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57082cf1-6992-4d2f-b5d0-01714fdc9734 None admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:46.185387 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57082cf1-6992-4d2f-b5d0-01714fdc9734 None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:46.185587 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57082cf1-6992-4d2f-b5d0-01714fdc9734 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:46.186242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57082cf1-6992-4d2f-b5d0-01714fdc9734 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:46.187759 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57082cf1-6992-4d2f-b5d0-01714fdc9734 None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:46.200831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application [None req-57082cf1-6992-4d2f-b5d0-01714fdc9734 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 18:30:46.200831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:46.200831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 18:30:46.200831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 18:30:46.200831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 18:30:46.200831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 18:30:46.200831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 18:30:46.200831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:30:46.200831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 18:30:46.200831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:46.200831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 18:30:46.200831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:46.200831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 18:30:46.200831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 18:30:46.200831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 18:30:46.200831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 18:30:46.200831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 18:30:46.202037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 18:30:46.202037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 18:30:46.202037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 18:30:46.202037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 18:30:46.202037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 18:30:46.202037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 18:30:46.202037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 18:30:46.202037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 18:30:46.202037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 18:30:46.202037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 18:30:46.202037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 18:30:46.202037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 18:30:46.202037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 18:30:46.202037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 18:30:46.202037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 18:30:46.202037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 18:30:46.203290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 18:30:46.203290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application with Lock( Jun 27 18:30:46.203290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 18:30:46.203290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self._enter() Jun 27 18:30:46.203290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 18:30:46.203290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 18:30:46.203290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 18:30:46.203290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application return self.creator() Jun 27 18:30:46.203290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 18:30:46.203290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application created_value = creator( Jun 27 18:30:46.203290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 18:30:46.203290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 18:30:46.203290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 18:30:46.203290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: ERROR keystone.server.flask.application Jun 27 18:30:46.203290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 667/1304] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Jun 27 18:30:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 18:30:46.208210 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba00f81-3683-4abc-981c-7c3230e39aae None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:46.285054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:46.308966 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba00f81-3683-4abc-981c-7c3230e39aae None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:46.342924 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57f9b4-e93d-4006-9185-9c59755f2618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:46.343573 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57f9b4-e93d-4006-9185-9c59755f2618 None admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:46.343853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57f9b4-e93d-4006-9185-9c59755f2618 None admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:46.344125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57f9b4-e93d-4006-9185-9c59755f2618 None admin] PATH_INFO: `/v3/services` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:46.344819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57f9b4-e93d-4006-9185-9c59755f2618 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:30:46.345120 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57f9b4-e93d-4006-9185-9c59755f2618 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:46.346571 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57f9b4-e93d-4006-9185-9c59755f2618 None admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:46.352858 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 638/1305] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Jun 27 18:30:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:30:46.358790 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57082cf1-6992-4d2f-b5d0-01714fdc9734 None admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:30:46.461613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57082cf1-6992-4d2f-b5d0-01714fdc9734 None admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:30:46.501267 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe28bcf-16eb-4053-8554-da79fa9208ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:30:46.502018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe28bcf-16eb-4053-8554-da79fa9208ce None admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:30:46.502345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe28bcf-16eb-4053-8554-da79fa9208ce None admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:30:46.502771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe28bcf-16eb-4053-8554-da79fa9208ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:30:46.503606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe28bcf-16eb-4053-8554-da79fa9208ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:30:46.505216 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe28bcf-16eb-4053-8554-da79fa9208ce None admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:30:46.587297 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 668/1306] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Jun 27 18:30:46 2022] POST /identity/v3/endpoints => generated 331 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:30:56.361440 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=30 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:30:56.582146 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:36:34.013637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1acd7-841d-40d1-aa94-5c435a7912a3 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:34.014173 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1acd7-841d-40d1-aa94-5c435a7912a3 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:34.014173 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1acd7-841d-40d1-aa94-5c435a7912a3 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:34.015507 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 639/1307] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:36:34.021271 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f269ff3-8773-4552-8d7d-c8a8246a2347 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:34.021647 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f269ff3-8773-4552-8d7d-c8a8246a2347 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:34.021808 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f269ff3-8773-4552-8d7d-c8a8246a2347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:34.101722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-1f269ff3-8773-4552-8d7d-c8a8246a2347 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:36:34.101722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-1f269ff3-8773-4552-8d7d-c8a8246a2347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:34.223241 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 669/1308] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:36:34.263117 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c0fd7-60b5-411b-b7fc-49cafbc0df62 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:34.263406 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c0fd7-60b5-411b-b7fc-49cafbc0df62 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:34.263738 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c0fd7-60b5-411b-b7fc-49cafbc0df62 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:34.265201 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 640/1309] 10.222.0.15 () {54 vars in 964 bytes} [Mon Jun 27 18:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:36:34.274701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd0a79-5339-4d86-8433-538d67c23510 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:34.275298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd0a79-5339-4d86-8433-538d67c23510 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:34.275755 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd0a79-5339-4d86-8433-538d67c23510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:34.375260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-4ccd0a79-5339-4d86-8433-538d67c23510 None None] MFA Rules not processed for user `dbe621916dbe4f1a8686097fe2bdf762`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:36:34.376722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-4ccd0a79-5339-4d86-8433-538d67c23510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:34.399903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:36:34.466577 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 670/1310] 10.222.0.15 () {58 vars in 1061 bytes} [Mon Jun 27 18:36:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:36:34.471329 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c0fd7-60b5-411b-b7fc-49cafbc0df62 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:36:34.473926 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-d96c0fd7-60b5-411b-b7fc-49cafbc0df62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:34.593871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c0fd7-60b5-411b-b7fc-49cafbc0df62 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:36:34.671725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdaaeed-f749-4a57-8a06-efd8549842af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:36:34.672427 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdaaeed-f749-4a57-8a06-efd8549842af service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:34.672687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdaaeed-f749-4a57-8a06-efd8549842af service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:34.672962 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdaaeed-f749-4a57-8a06-efd8549842af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:34.674640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-3cdaaeed-f749-4a57-8a06-efd8549842af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:34.731806 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:36:34.794255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdaaeed-f749-4a57-8a06-efd8549842af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:36:34.795720 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdaaeed-f749-4a57-8a06-efd8549842af service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:36:34.814884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 641/1311] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:36:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:36:37.170459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64423e92-f565-4981-af00-b786df7151df None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:37.170973 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64423e92-f565-4981-af00-b786df7151df None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:37.170973 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64423e92-f565-4981-af00-b786df7151df None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:37.174435 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 671/1312] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:36:37.178759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ba4b1-6ca9-43e4-a282-bc5b2be77f83 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:37.179459 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ba4b1-6ca9-43e4-a282-bc5b2be77f83 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:37.179750 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ba4b1-6ca9-43e4-a282-bc5b2be77f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:37.263250 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-c27ba4b1-6ca9-43e4-a282-bc5b2be77f83 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:36:37.264466 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-c27ba4b1-6ca9-43e4-a282-bc5b2be77f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:37.367862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 642/1313] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:36:37.406574 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64423e92-f565-4981-af00-b786df7151df None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:36:37.408346 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-64423e92-f565-4981-af00-b786df7151df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:37.522625 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64423e92-f565-4981-af00-b786df7151df None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:36:37.608998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:36:37.611725 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb6453-dc46-4401-8948-7d400ed526ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:36:37.612739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb6453-dc46-4401-8948-7d400ed526ad service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:37.613559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb6453-dc46-4401-8948-7d400ed526ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:37.613835 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb6453-dc46-4401-8948-7d400ed526ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:37.615607 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-53eb6453-dc46-4401-8948-7d400ed526ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:37.757388 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb6453-dc46-4401-8948-7d400ed526ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:36:37.758909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb6453-dc46-4401-8948-7d400ed526ad service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:36:37.780377 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 672/1314] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:36:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:36:40.343228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fa0db-291c-4971-8364-109f8c6d12fd None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:40.343799 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fa0db-291c-4971-8364-109f8c6d12fd None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:40.343799 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fa0db-291c-4971-8364-109f8c6d12fd None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:40.345198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 643/1315] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:36:40.350362 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4110f-52a2-4d02-b93c-2a20f752b612 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:40.350798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4110f-52a2-4d02-b93c-2a20f752b612 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:40.350967 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4110f-52a2-4d02-b93c-2a20f752b612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:40.414492 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-ffe4110f-52a2-4d02-b93c-2a20f752b612 None None] MFA Rules not processed for user `69a2de33463a462ca1305bf7bea04a37`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:36:40.415617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ffe4110f-52a2-4d02-b93c-2a20f752b612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:40.466833 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:36:40.509968 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 673/1316] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:36:40 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:36:40.562567 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078aa4d-faea-4441-a370-f0657a0cb763 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:40.562857 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078aa4d-faea-4441-a370-f0657a0cb763 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:40.563139 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078aa4d-faea-4441-a370-f0657a0cb763 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:40.564553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 644/1317] 10.222.0.15 () {54 vars in 964 bytes} [Mon Jun 27 18:36:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:36:40.571037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a888e7-2394-4802-944f-36284d76cd65 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:40.571329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a888e7-2394-4802-944f-36284d76cd65 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:40.571609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a888e7-2394-4802-944f-36284d76cd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:40.644768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-72a888e7-2394-4802-944f-36284d76cd65 None None] MFA Rules not processed for user `dbe621916dbe4f1a8686097fe2bdf762`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:36:40.645956 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-72a888e7-2394-4802-944f-36284d76cd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:40.720518 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 674/1318] 10.222.0.15 () {58 vars in 1061 bytes} [Mon Jun 27 18:36:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:36:40.729640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078aa4d-faea-4441-a370-f0657a0cb763 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:36:40.731248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-c078aa4d-faea-4441-a370-f0657a0cb763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:40.742155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:36:40.836511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078aa4d-faea-4441-a370-f0657a0cb763 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:36:40.908053 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691a587-6b1d-46e6-9c98-76c26be76631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:36:40.908681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691a587-6b1d-46e6-9c98-76c26be76631 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:40.908958 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691a587-6b1d-46e6-9c98-76c26be76631 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:40.909227 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691a587-6b1d-46e6-9c98-76c26be76631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:40.911893 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-8691a587-6b1d-46e6-9c98-76c26be76631 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:40.973774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:36:41.020366 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8691a587-6b1d-46e6-9c98-76c26be76631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:36:41.021784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8691a587-6b1d-46e6-9c98-76c26be76631 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:36:41.040065 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 645/1319] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:36:40 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:36:44.254119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecca2a-5183-47df-b5a5-6257d2e60986 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:44.254719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecca2a-5183-47df-b5a5-6257d2e60986 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:44.254719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecca2a-5183-47df-b5a5-6257d2e60986 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:44.256177 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 675/1320] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:36:44.262261 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d18fa6-e3ea-4217-9593-996f504aedd2 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:44.262764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d18fa6-e3ea-4217-9593-996f504aedd2 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:44.262860 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d18fa6-e3ea-4217-9593-996f504aedd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:44.331348 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-a0d18fa6-e3ea-4217-9593-996f504aedd2 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:36:44.333590 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-a0d18fa6-e3ea-4217-9593-996f504aedd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:44.442943 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 646/1321] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:36:44 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:36:44.484468 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecca2a-5183-47df-b5a5-6257d2e60986 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:36:44.486031 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-5cecca2a-5183-47df-b5a5-6257d2e60986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:44.538673 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:36:44.599228 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecca2a-5183-47df-b5a5-6257d2e60986 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:36:44.679026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780dd47-b0e3-48bc-8ced-dacf84c9a5c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:36:44.679664 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780dd47-b0e3-48bc-8ced-dacf84c9a5c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:44.679943 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780dd47-b0e3-48bc-8ced-dacf84c9a5c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:44.680215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780dd47-b0e3-48bc-8ced-dacf84c9a5c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:44.681883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-0780dd47-b0e3-48bc-8ced-dacf84c9a5c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:44.779309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:36:44.792830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780dd47-b0e3-48bc-8ced-dacf84c9a5c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:36:44.794284 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780dd47-b0e3-48bc-8ced-dacf84c9a5c5 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:36:44.813543 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 676/1322] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:36:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:36:47.904451 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1973b-c875-4e2f-9b8d-54d7f6f06f05 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:47.905011 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1973b-c875-4e2f-9b8d-54d7f6f06f05 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:47.905011 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1973b-c875-4e2f-9b8d-54d7f6f06f05 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:47.906417 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 647/1323] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:36:47.911867 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23785f-1ee0-4f5b-8dc3-db40b2d9dcca None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:47.912312 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23785f-1ee0-4f5b-8dc3-db40b2d9dcca None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:47.912565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23785f-1ee0-4f5b-8dc3-db40b2d9dcca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:47.978320 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-1d23785f-1ee0-4f5b-8dc3-db40b2d9dcca None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:36:47.979565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-1d23785f-1ee0-4f5b-8dc3-db40b2d9dcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:48.079093 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 677/1324] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:36:48.117295 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1973b-c875-4e2f-9b8d-54d7f6f06f05 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:36:48.118993 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-72a1973b-c875-4e2f-9b8d-54d7f6f06f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:48.140506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:36:48.233365 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a1973b-c875-4e2f-9b8d-54d7f6f06f05 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:36:48.314932 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf203a-1192-40a9-bd0b-85d9f7331337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:36:48.315576 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf203a-1192-40a9-bd0b-85d9f7331337 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:48.316477 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf203a-1192-40a9-bd0b-85d9f7331337 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:48.316770 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf203a-1192-40a9-bd0b-85d9f7331337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:48.318470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-93bf203a-1192-40a9-bd0b-85d9f7331337 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:48.399947 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:36:48.439108 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf203a-1192-40a9-bd0b-85d9f7331337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:36:48.440542 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf203a-1192-40a9-bd0b-85d9f7331337 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:36:48.458408 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 648/1325] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:36:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:36:50.558853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497fd64-e457-42e2-b313-18856ca68f5b None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:50.559411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497fd64-e457-42e2-b313-18856ca68f5b None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:50.559411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497fd64-e457-42e2-b313-18856ca68f5b None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:50.560740 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 678/1326] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:36:50.566722 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef61e2a-0297-4b03-9f30-181f377400d0 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:50.567175 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef61e2a-0297-4b03-9f30-181f377400d0 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:50.567268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef61e2a-0297-4b03-9f30-181f377400d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:50.634553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-eef61e2a-0297-4b03-9f30-181f377400d0 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:36:50.635712 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-eef61e2a-0297-4b03-9f30-181f377400d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:50.738493 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 649/1327] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:36:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:36:50.773480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497fd64-e457-42e2-b313-18856ca68f5b None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:36:50.775049 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-7497fd64-e457-42e2-b313-18856ca68f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:50.835165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:36:50.879118 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497fd64-e457-42e2-b313-18856ca68f5b None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:36:50.952056 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b02c19-8880-4f9c-baee-cc5ccde6f010 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:36:50.952700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b02c19-8880-4f9c-baee-cc5ccde6f010 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:50.952980 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b02c19-8880-4f9c-baee-cc5ccde6f010 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:50.953254 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b02c19-8880-4f9c-baee-cc5ccde6f010 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:50.954931 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-26b02c19-8880-4f9c-baee-cc5ccde6f010 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:51.065328 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:36:51.068114 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b02c19-8880-4f9c-baee-cc5ccde6f010 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:36:51.069586 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b02c19-8880-4f9c-baee-cc5ccde6f010 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:36:51.089415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 679/1328] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:36:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:36:53.193959 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a4756-0632-4c44-bc17-5cfff970af84 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:53.194533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a4756-0632-4c44-bc17-5cfff970af84 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:53.194533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a4756-0632-4c44-bc17-5cfff970af84 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:53.196051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 650/1329] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:36:53.200998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212347e-c32e-407f-b4ef-3350ccf0fa0a None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:53.201402 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212347e-c32e-407f-b4ef-3350ccf0fa0a None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:53.201557 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212347e-c32e-407f-b4ef-3350ccf0fa0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:53.268165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-b212347e-c32e-407f-b4ef-3350ccf0fa0a None None] MFA Rules not processed for user `69a2de33463a462ca1305bf7bea04a37`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:36:53.269760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-b212347e-c32e-407f-b4ef-3350ccf0fa0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:53.370138 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 680/1330] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:36:53 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:36:53.407905 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a4756-0632-4c44-bc17-5cfff970af84 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:36:53.410055 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-9a6a4756-0632-4c44-bc17-5cfff970af84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:53.445463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:36:53.523033 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a4756-0632-4c44-bc17-5cfff970af84 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:36:53.598503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58cc978-2599-4c6a-be9e-3be539d65157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:36:53.599140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cc978-2599-4c6a-be9e-3be539d65157 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:53.599418 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cc978-2599-4c6a-be9e-3be539d65157 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:53.599689 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cc978-2599-4c6a-be9e-3be539d65157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:53.601329 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-b58cc978-2599-4c6a-be9e-3be539d65157 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:53.682361 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:36:53.714322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58cc978-2599-4c6a-be9e-3be539d65157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:36:53.715770 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58cc978-2599-4c6a-be9e-3be539d65157 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:36:53.734175 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 651/1331] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:36:53 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:36:56.229491 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06217e16-3b0a-4f0e-b8a6-dc620972324c None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:56.230052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06217e16-3b0a-4f0e-b8a6-dc620972324c None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:56.230052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06217e16-3b0a-4f0e-b8a6-dc620972324c None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:56.231617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 681/1332] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:36:56.236771 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec0a93-46e3-43a4-a98a-5498e1580298 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:56.237168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec0a93-46e3-43a4-a98a-5498e1580298 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:56.237328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec0a93-46e3-43a4-a98a-5498e1580298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:56.303372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-71ec0a93-46e3-43a4-a98a-5498e1580298 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:36:56.304602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-71ec0a93-46e3-43a4-a98a-5498e1580298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:56.400515 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 652/1333] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:36:56.436803 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06217e16-3b0a-4f0e-b8a6-dc620972324c None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:36:56.438333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-06217e16-3b0a-4f0e-b8a6-dc620972324c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:56.507539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:36:56.541614 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06217e16-3b0a-4f0e-b8a6-dc620972324c None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:36:56.614292 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f84e46-a0ce-4cb8-9b51-ee8450faa890 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:36:56.615029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f84e46-a0ce-4cb8-9b51-ee8450faa890 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:56.615311 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f84e46-a0ce-4cb8-9b51-ee8450faa890 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:56.615587 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f84e46-a0ce-4cb8-9b51-ee8450faa890 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:56.617238 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-d5f84e46-a0ce-4cb8-9b51-ee8450faa890 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:56.726758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f84e46-a0ce-4cb8-9b51-ee8450faa890 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:36:56.728115 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f84e46-a0ce-4cb8-9b51-ee8450faa890 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:36:56.738792 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:36:56.747851 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 682/1334] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:36:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:36:58.819988 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6378a3-c8b9-4410-ae28-8df1cc616e48 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:58.820531 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6378a3-c8b9-4410-ae28-8df1cc616e48 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:58.820629 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6378a3-c8b9-4410-ae28-8df1cc616e48 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:58.822013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 653/1335] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:36:58.827325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efab29a-8124-454e-b67e-558e70d16d77 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:58.827818 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efab29a-8124-454e-b67e-558e70d16d77 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:58.827927 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efab29a-8124-454e-b67e-558e70d16d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:58.896371 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-0efab29a-8124-454e-b67e-558e70d16d77 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:36:58.897580 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-0efab29a-8124-454e-b67e-558e70d16d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:58.993199 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 683/1336] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:36:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:36:59.033304 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6378a3-c8b9-4410-ae28-8df1cc616e48 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:36:59.034978 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-5f6378a3-c8b9-4410-ae28-8df1cc616e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:59.074812 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:36:59.138056 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6378a3-c8b9-4410-ae28-8df1cc616e48 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:36:59.209322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110aefb-dbef-445d-b89e-93af9cf54fd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:36:59.209951 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110aefb-dbef-445d-b89e-93af9cf54fd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:36:59.210226 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110aefb-dbef-445d-b89e-93af9cf54fd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:36:59.210532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110aefb-dbef-445d-b89e-93af9cf54fd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:36:59.212174 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-0110aefb-dbef-445d-b89e-93af9cf54fd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:36:59.292490 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:36:59.319004 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110aefb-dbef-445d-b89e-93af9cf54fd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:36:59.320422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110aefb-dbef-445d-b89e-93af9cf54fd1 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:36:59.339351 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 654/1337] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:36:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:37:01.902984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74919b-81e3-445b-bac1-226e6a123e8b None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:01.903542 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74919b-81e3-445b-bac1-226e6a123e8b None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:01.903542 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74919b-81e3-445b-bac1-226e6a123e8b None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:01.904950 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 684/1338] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:01.910275 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323c619-61ae-4a51-ae9c-edb3a4d0667e None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:01.910742 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323c619-61ae-4a51-ae9c-edb3a4d0667e None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:01.910993 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323c619-61ae-4a51-ae9c-edb3a4d0667e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:01.975318 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-0323c619-61ae-4a51-ae9c-edb3a4d0667e None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:01.976521 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-0323c619-61ae-4a51-ae9c-edb3a4d0667e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:02.070609 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 655/1339] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:37:01 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:37:02.105637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74919b-81e3-445b-bac1-226e6a123e8b None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:37:02.107198 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ae74919b-81e3-445b-bac1-226e6a123e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:02.187911 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:37:02.215939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74919b-81e3-445b-bac1-226e6a123e8b None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:37:02.288411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2584e9-2a26-4b40-be68-860af01a87f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:37:02.289046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2584e9-2a26-4b40-be68-860af01a87f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:02.289330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2584e9-2a26-4b40-be68-860af01a87f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:02.289605 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2584e9-2a26-4b40-be68-860af01a87f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:02.291274 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-9b2584e9-2a26-4b40-be68-860af01a87f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:02.396504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2584e9-2a26-4b40-be68-860af01a87f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:37:02.397935 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2584e9-2a26-4b40-be68-860af01a87f0 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:37:02.416643 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 685/1340] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:37:02 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:37:06.635941 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b3bad-4e4a-4cda-96a0-58e37daf245e None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:06.636464 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b3bad-4e4a-4cda-96a0-58e37daf245e None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:06.636571 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b3bad-4e4a-4cda-96a0-58e37daf245e None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:06.637928 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 656/1341] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:06.643425 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947e088-d83d-4507-a091-45358366a380 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:06.644301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947e088-d83d-4507-a091-45358366a380 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:06.644585 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947e088-d83d-4507-a091-45358366a380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:06.651247 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:37:06.726621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-6947e088-d83d-4507-a091-45358366a380 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:06.727997 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-6947e088-d83d-4507-a091-45358366a380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:06.838263 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 686/1342] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:37:06.877051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b3bad-4e4a-4cda-96a0-58e37daf245e None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:37:06.880006 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-e19b3bad-4e4a-4cda-96a0-58e37daf245e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:06.932027 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:37:06.977723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b3bad-4e4a-4cda-96a0-58e37daf245e None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:37:06.981477 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858579e-7172-47f8-a481-17216d5f7155 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:37:06.982111 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858579e-7172-47f8-a481-17216d5f7155 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:06.982405 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858579e-7172-47f8-a481-17216d5f7155 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:06.982686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858579e-7172-47f8-a481-17216d5f7155 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:06.992438 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858579e-7172-47f8-a481-17216d5f7155 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:37:06.993850 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858579e-7172-47f8-a481-17216d5f7155 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:37:07.003221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 657/1343] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:37:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:37:10.025354 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a210f-f0d5-48e8-be89-b43300fc4b76 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:10.025879 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a210f-f0d5-48e8-be89-b43300fc4b76 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:10.025879 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a210f-f0d5-48e8-be89-b43300fc4b76 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:10.027248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 687/1344] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:10.032590 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffbf66-1bf3-416b-85e2-d42fe3722c06 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:10.033001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffbf66-1bf3-416b-85e2-d42fe3722c06 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:10.033253 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffbf66-1bf3-416b-85e2-d42fe3722c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:10.061194 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-87ffbf66-1bf3-416b-85e2-d42fe3722c06 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:10.062369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-87ffbf66-1bf3-416b-85e2-d42fe3722c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:10.141408 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 658/1345] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:37:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:37:10.178651 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a210f-f0d5-48e8-be89-b43300fc4b76 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:37:10.189430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a210f-f0d5-48e8-be89-b43300fc4b76 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:37:10.193901 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00d8a7-3150-4fe9-87c1-d01f58ded08f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:37:10.194551 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00d8a7-3150-4fe9-87c1-d01f58ded08f service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:10.194833 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00d8a7-3150-4fe9-87c1-d01f58ded08f service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:10.195104 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00d8a7-3150-4fe9-87c1-d01f58ded08f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:10.203870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00d8a7-3150-4fe9-87c1-d01f58ded08f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:37:10.205280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00d8a7-3150-4fe9-87c1-d01f58ded08f service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:37:10.215442 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 688/1346] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:37:10 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:37:14.496878 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99917953-30af-4467-ba61-93034af414b2 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:14.497412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99917953-30af-4467-ba61-93034af414b2 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:14.497576 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99917953-30af-4467-ba61-93034af414b2 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:14.499337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 659/1347] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:14.504871 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06b0f2-ee57-485e-98e3-d8eb6dd4053a None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:14.505298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06b0f2-ee57-485e-98e3-d8eb6dd4053a None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:14.505548 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06b0f2-ee57-485e-98e3-d8eb6dd4053a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:14.523309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-ab06b0f2-ee57-485e-98e3-d8eb6dd4053a None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:14.524531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ab06b0f2-ee57-485e-98e3-d8eb6dd4053a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:14.532567 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 689/1348] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:37:14.571443 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99917953-30af-4467-ba61-93034af414b2 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:37:14.573253 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-99917953-30af-4467-ba61-93034af414b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:14.586703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99917953-30af-4467-ba61-93034af414b2 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:37:14.590352 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab07ba-2c74-4fbc-9fd9-8bdb9ab01ec9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:37:14.591006 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab07ba-2c74-4fbc-9fd9-8bdb9ab01ec9 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:14.591285 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab07ba-2c74-4fbc-9fd9-8bdb9ab01ec9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:14.591567 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab07ba-2c74-4fbc-9fd9-8bdb9ab01ec9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:14.601267 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ab07ba-2c74-4fbc-9fd9-8bdb9ab01ec9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:37:14.602729 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ab07ba-2c74-4fbc-9fd9-8bdb9ab01ec9 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:37:14.612734 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 660/1349] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:37:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:37:19.651032 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda4bf4-a97c-4575-8b07-bd8eab51fa9f None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:19.651565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda4bf4-a97c-4575-8b07-bd8eab51fa9f None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:19.651565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda4bf4-a97c-4575-8b07-bd8eab51fa9f None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:19.652950 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 690/1350] 10.222.0.15 () {54 vars in 921 bytes} [Mon Jun 27 18:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:19.658372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57086e3d-3ec4-4d09-99a1-218cc122d92a None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:19.659574 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57086e3d-3ec4-4d09-99a1-218cc122d92a None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:19.659574 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57086e3d-3ec4-4d09-99a1-218cc122d92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:19.756584 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-57086e3d-3ec4-4d09-99a1-218cc122d92a None None] MFA Rules not processed for user `3cd6f0a1286243e4add692db01028205`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:19.758121 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-57086e3d-3ec4-4d09-99a1-218cc122d92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:19.844676 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:37:19.902659 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 661/1351] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Jun 27 18:37:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:37:24.120223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a867a-2da3-42a3-8dbc-c5ae639afe06 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:24.120806 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a867a-2da3-42a3-8dbc-c5ae639afe06 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:24.120806 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a867a-2da3-42a3-8dbc-c5ae639afe06 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:24.122316 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 691/1352] 10.222.0.15 () {54 vars in 921 bytes} [Mon Jun 27 18:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:24.127348 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d86a31-2cdc-4167-bc77-b90410a9ce33 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:24.128831 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d86a31-2cdc-4167-bc77-b90410a9ce33 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:24.128831 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d86a31-2cdc-4167-bc77-b90410a9ce33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:24.147242 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-06d86a31-2cdc-4167-bc77-b90410a9ce33 None None] MFA Rules not processed for user `3cd6f0a1286243e4add692db01028205`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:24.148573 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-06d86a31-2cdc-4167-bc77-b90410a9ce33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:24.157895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 662/1353] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Jun 27 18:37:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:37:24.518959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=72 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:37:25.749358 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649fb1e-d2cb-4751-9c34-a843c358b704 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:25.749941 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649fb1e-d2cb-4751-9c34-a843c358b704 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:25.750222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649fb1e-d2cb-4751-9c34-a843c358b704 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:25.751886 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 692/1354] 10.222.0.15 () {54 vars in 921 bytes} [Mon Jun 27 18:37:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:25.758093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9dc05c-2f32-4126-a9a0-a3a96efa7f52 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:25.758620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9dc05c-2f32-4126-a9a0-a3a96efa7f52 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:25.758814 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9dc05c-2f32-4126-a9a0-a3a96efa7f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:25.777872 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-ee9dc05c-2f32-4126-a9a0-a3a96efa7f52 None None] MFA Rules not processed for user `3cd6f0a1286243e4add692db01028205`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:25.779109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-ee9dc05c-2f32-4126-a9a0-a3a96efa7f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:25.787961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 663/1355] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Jun 27 18:37:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:37:29.293917 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6046728-faa2-4dd4-a46d-ad6e342babe3 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:29.294459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6046728-faa2-4dd4-a46d-ad6e342babe3 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:29.294459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6046728-faa2-4dd4-a46d-ad6e342babe3 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:29.296033 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 693/1356] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:29.301621 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea9a7c-6ee9-4637-a91d-9f665962d1f7 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:29.302106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea9a7c-6ee9-4637-a91d-9f665962d1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:29.302223 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea9a7c-6ee9-4637-a91d-9f665962d1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:29.327795 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-48ea9a7c-6ee9-4637-a91d-9f665962d1f7 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:29.328969 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-48ea9a7c-6ee9-4637-a91d-9f665962d1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:29.338660 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 664/1357] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:37:29.394043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b0ddd-ee90-4fc7-a26a-dc5efda4256a None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:29.394714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b0ddd-ee90-4fc7-a26a-dc5efda4256a None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:29.395160 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b0ddd-ee90-4fc7-a26a-dc5efda4256a None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:29.396936 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 694/1358] 10.222.0.15 () {54 vars in 961 bytes} [Mon Jun 27 18:37:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:29.406581 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6174b-55d2-4b83-a963-e52963fd1663 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:29.407089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6174b-55d2-4b83-a963-e52963fd1663 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:29.407528 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6174b-55d2-4b83-a963-e52963fd1663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:29.464096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-4da6174b-55d2-4b83-a963-e52963fd1663 None None] MFA Rules not processed for user `1dddd4efb44c4152bb4151f7715f2223`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:29.465245 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-4da6174b-55d2-4b83-a963-e52963fd1663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:29.557284 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 665/1359] 10.222.0.15 () {58 vars in 1058 bytes} [Mon Jun 27 18:37:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:37:29.566578 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b0ddd-ee90-4fc7-a26a-dc5efda4256a None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:37:29.590286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b0ddd-ee90-4fc7-a26a-dc5efda4256a None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:37:29.597587 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627db7fa-2b8e-43d6-8644-22ab19fad422 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:37:29.598238 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627db7fa-2b8e-43d6-8644-22ab19fad422 service nova] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:29.598593 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627db7fa-2b8e-43d6-8644-22ab19fad422 service nova] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:29.598874 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627db7fa-2b8e-43d6-8644-22ab19fad422 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:29.607978 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627db7fa-2b8e-43d6-8644-22ab19fad422 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:37:29.609607 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627db7fa-2b8e-43d6-8644-22ab19fad422 service nova] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:37:29.619968 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 695/1360] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:37:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:37:31.914172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f26b94-faa2-425b-b881-f50c7c0e1404 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:31.914735 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f26b94-faa2-425b-b881-f50c7c0e1404 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:31.914735 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f26b94-faa2-425b-b881-f50c7c0e1404 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:31.916268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 666/1361] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:31.921218 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce7991-8d93-495b-b680-39e2ac019453 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:31.921637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce7991-8d93-495b-b680-39e2ac019453 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:31.921773 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce7991-8d93-495b-b680-39e2ac019453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:31.940989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-35ce7991-8d93-495b-b680-39e2ac019453 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:31.942186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-35ce7991-8d93-495b-b680-39e2ac019453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:31.950731 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 696/1362] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:37:31.995628 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f26b94-faa2-425b-b881-f50c7c0e1404 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:37:32.010566 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f26b94-faa2-425b-b881-f50c7c0e1404 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:37:32.015807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddeea7c-7e1c-43e8-8739-9dec4153f65b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:37:32.016485 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddeea7c-7e1c-43e8-8739-9dec4153f65b service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:32.016755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddeea7c-7e1c-43e8-8739-9dec4153f65b service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:32.017033 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddeea7c-7e1c-43e8-8739-9dec4153f65b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:32.025927 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddeea7c-7e1c-43e8-8739-9dec4153f65b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:37:32.027547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddeea7c-7e1c-43e8-8739-9dec4153f65b service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:37:32.037556 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 667/1363] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:37:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:37:34.599857 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce39359-951c-476e-8071-bebab2868011 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:34.600386 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce39359-951c-476e-8071-bebab2868011 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:34.600481 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce39359-951c-476e-8071-bebab2868011 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:34.601781 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 697/1364] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:34.607250 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea69660-5fad-4544-88ab-6d7343e1eea5 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:34.607662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea69660-5fad-4544-88ab-6d7343e1eea5 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:34.607826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea69660-5fad-4544-88ab-6d7343e1eea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:34.625613 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-eea69660-5fad-4544-88ab-6d7343e1eea5 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:34.639317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-eea69660-5fad-4544-88ab-6d7343e1eea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:34.639317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 668/1365] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:37:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:37:34.681487 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082178a3-1b44-44ae-950a-50bc1e1faf7e None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:34.681873 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082178a3-1b44-44ae-950a-50bc1e1faf7e None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:34.682224 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082178a3-1b44-44ae-950a-50bc1e1faf7e None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:34.683725 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 698/1366] 10.222.0.15 () {54 vars in 961 bytes} [Mon Jun 27 18:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:34.689023 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a185e25-dcb8-46db-aded-f3ad463a7754 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:34.689309 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a185e25-dcb8-46db-aded-f3ad463a7754 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:34.690106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a185e25-dcb8-46db-aded-f3ad463a7754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:34.708284 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-4a185e25-dcb8-46db-aded-f3ad463a7754 None None] MFA Rules not processed for user `1dddd4efb44c4152bb4151f7715f2223`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:34.709499 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-4a185e25-dcb8-46db-aded-f3ad463a7754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:34.718640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 669/1367] 10.222.0.15 () {58 vars in 1058 bytes} [Mon Jun 27 18:37:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:37:34.727417 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082178a3-1b44-44ae-950a-50bc1e1faf7e None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:37:34.742856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082178a3-1b44-44ae-950a-50bc1e1faf7e None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:37:34.750150 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f977381-ebbd-4ab5-b6ed-a0a3f057068d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:37:34.750818 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f977381-ebbd-4ab5-b6ed-a0a3f057068d service nova] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:34.751104 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f977381-ebbd-4ab5-b6ed-a0a3f057068d service nova] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:34.751379 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f977381-ebbd-4ab5-b6ed-a0a3f057068d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:34.760672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f977381-ebbd-4ab5-b6ed-a0a3f057068d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:37:34.762280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f977381-ebbd-4ab5-b6ed-a0a3f057068d service nova] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:37:34.780236 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 699/1368] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:37:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:37:37.107598 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb06f2-01fa-4ba0-8fb0-9907f103e072 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:37.108128 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb06f2-01fa-4ba0-8fb0-9907f103e072 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:37.108235 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb06f2-01fa-4ba0-8fb0-9907f103e072 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:37.109528 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 670/1369] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:37.115960 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b118b8e-0700-4975-9a72-b2c9314229ff None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:37.116471 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b118b8e-0700-4975-9a72-b2c9314229ff None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:37.116575 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b118b8e-0700-4975-9a72-b2c9314229ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:37.134098 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-3b118b8e-0700-4975-9a72-b2c9314229ff None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:37.135340 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-3b118b8e-0700-4975-9a72-b2c9314229ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:37.143594 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 700/1370] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:37:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:37:37.187154 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb06f2-01fa-4ba0-8fb0-9907f103e072 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:37:37.202407 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb06f2-01fa-4ba0-8fb0-9907f103e072 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:37:37.208580 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba765366-84bc-4d5a-b8be-18b594c94e75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:37:37.209226 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba765366-84bc-4d5a-b8be-18b594c94e75 service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:37.209509 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba765366-84bc-4d5a-b8be-18b594c94e75 service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:37.209808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba765366-84bc-4d5a-b8be-18b594c94e75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:37.218415 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba765366-84bc-4d5a-b8be-18b594c94e75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:37:37.220007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba765366-84bc-4d5a-b8be-18b594c94e75 service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:37:37.230041 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 671/1371] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:37:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:37:37.557688 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b39a09-7c82-4fd1-9aaf-a8c984d4c162 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:37.557978 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b39a09-7c82-4fd1-9aaf-a8c984d4c162 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:37.558502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b39a09-7c82-4fd1-9aaf-a8c984d4c162 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:37.560311 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 701/1372] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:37:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:37.566107 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25027439-e7c1-46e2-a416-f3ca7bd3d93f None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:37.566732 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25027439-e7c1-46e2-a416-f3ca7bd3d93f None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:37.567074 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25027439-e7c1-46e2-a416-f3ca7bd3d93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:37.585333 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-25027439-e7c1-46e2-a416-f3ca7bd3d93f None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:37.586563 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-25027439-e7c1-46e2-a416-f3ca7bd3d93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:37.594365 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 672/1373] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:37:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:37:37.936204 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348736c-8079-425f-b390-0b6917b1854b None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:37.936497 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348736c-8079-425f-b390-0b6917b1854b None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:37.936775 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348736c-8079-425f-b390-0b6917b1854b None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:37.942119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 702/1374] 10.222.0.15 () {54 vars in 963 bytes} [Mon Jun 27 18:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:37.943844 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e19d9b-ddae-4f8e-a418-b153af870130 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:37.944140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e19d9b-ddae-4f8e-a418-b153af870130 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:37.944424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e19d9b-ddae-4f8e-a418-b153af870130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:37.951211 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:37:37.996657 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-23e19d9b-ddae-4f8e-a418-b153af870130 None None] MFA Rules not processed for user `c4f6f8e0a0cb4c9a9dd0d99fc0022235`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:37.997934 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-23e19d9b-ddae-4f8e-a418-b153af870130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:38.064203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 673/1375] 10.222.0.15 () {58 vars in 1060 bytes} [Mon Jun 27 18:37:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:37:38.072239 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348736c-8079-425f-b390-0b6917b1854b None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:37:38.084228 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348736c-8079-425f-b390-0b6917b1854b None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:37:38.087876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fa330-82f3-4ae3-a625-c09f543f8ebf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:37:38.088606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fa330-82f3-4ae3-a625-c09f543f8ebf service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:38.088965 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fa330-82f3-4ae3-a625-c09f543f8ebf service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:38.089316 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fa330-82f3-4ae3-a625-c09f543f8ebf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:38.099894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82fa330-82f3-4ae3-a625-c09f543f8ebf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:37:38.101728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82fa330-82f3-4ae3-a625-c09f543f8ebf service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:37:38.112414 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 703/1376] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:37:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:37:39.439025 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b4d2e-a802-4dc6-a6c8-7673a654f4c4 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:39.440524 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b4d2e-a802-4dc6-a6c8-7673a654f4c4 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:39.440643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b4d2e-a802-4dc6-a6c8-7673a654f4c4 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:39.442096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 674/1377] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:37:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:39.447435 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf4056-9e2c-41fa-9742-5b704acf69d2 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:39.447843 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf4056-9e2c-41fa-9742-5b704acf69d2 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:39.448121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf4056-9e2c-41fa-9742-5b704acf69d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:39.466625 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-5cbf4056-9e2c-41fa-9742-5b704acf69d2 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:39.467832 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-5cbf4056-9e2c-41fa-9742-5b704acf69d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:39.476058 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 704/1378] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:37:39.521135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b4d2e-a802-4dc6-a6c8-7673a654f4c4 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:37:39.536081 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b4d2e-a802-4dc6-a6c8-7673a654f4c4 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:37:39.541570 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a805305-ff61-43cf-8387-c43e21778b4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:37:39.542210 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a805305-ff61-43cf-8387-c43e21778b4e service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:39.542541 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a805305-ff61-43cf-8387-c43e21778b4e service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:39.542822 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a805305-ff61-43cf-8387-c43e21778b4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:39.551993 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a805305-ff61-43cf-8387-c43e21778b4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:37:39.553767 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a805305-ff61-43cf-8387-c43e21778b4e service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:37:39.563198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 675/1379] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:37:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:37:41.801733 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dfe7b-ea48-4153-8370-79a20fb83e10 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:41.802603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dfe7b-ea48-4153-8370-79a20fb83e10 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:41.803058 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dfe7b-ea48-4153-8370-79a20fb83e10 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:41.808698 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 705/1380] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:37:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:41.811404 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1215b-8fb7-46c7-b12c-28f96d469abe None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:41.811404 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1215b-8fb7-46c7-b12c-28f96d469abe None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:41.811404 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1215b-8fb7-46c7-b12c-28f96d469abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:41.833183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-86a1215b-8fb7-46c7-b12c-28f96d469abe None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:41.834670 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-86a1215b-8fb7-46c7-b12c-28f96d469abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:41.844051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 676/1381] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:37:41.893164 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6dfe7b-ea48-4153-8370-79a20fb83e10 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:37:41.908226 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6dfe7b-ea48-4153-8370-79a20fb83e10 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:37:41.913692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c6b31-3118-495c-b3ea-75cb61ea06a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:37:41.914324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c6b31-3118-495c-b3ea-75cb61ea06a4 service nova] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:41.914700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c6b31-3118-495c-b3ea-75cb61ea06a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:41.914977 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c6b31-3118-495c-b3ea-75cb61ea06a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:41.923229 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c6b31-3118-495c-b3ea-75cb61ea06a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:37:41.925369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c6b31-3118-495c-b3ea-75cb61ea06a4 service nova] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:37:41.935218 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 706/1382] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:37:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:37:44.183398 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663cf61-42d1-4fe8-b77b-ded2fd3af209 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:44.183828 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663cf61-42d1-4fe8-b77b-ded2fd3af209 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:44.184104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663cf61-42d1-4fe8-b77b-ded2fd3af209 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:44.190236 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 677/1383] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:37:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:44.192263 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecc07e-056a-48c8-8100-6a284bb012fa None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:44.192728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecc07e-056a-48c8-8100-6a284bb012fa None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:44.192848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecc07e-056a-48c8-8100-6a284bb012fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:44.219178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-abecc07e-056a-48c8-8100-6a284bb012fa None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:44.220616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-abecc07e-056a-48c8-8100-6a284bb012fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:44.234978 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 707/1384] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:37:44.277435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663cf61-42d1-4fe8-b77b-ded2fd3af209 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:37:44.293689 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3663cf61-42d1-4fe8-b77b-ded2fd3af209 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:37:44.300897 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a728a88-a1e7-462f-831d-db10dab7640d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:37:44.301537 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a728a88-a1e7-462f-831d-db10dab7640d service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:44.301817 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a728a88-a1e7-462f-831d-db10dab7640d service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:44.302090 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a728a88-a1e7-462f-831d-db10dab7640d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:44.312197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a728a88-a1e7-462f-831d-db10dab7640d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:37:44.313771 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a728a88-a1e7-462f-831d-db10dab7640d service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:37:44.324446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 678/1385] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:37:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:37:46.425805 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db7133-0b1d-437d-9b5e-fbe29b05b125 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:46.426514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db7133-0b1d-437d-9b5e-fbe29b05b125 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:46.426514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db7133-0b1d-437d-9b5e-fbe29b05b125 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:46.427723 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 708/1386] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:46.432922 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed771d-bd2c-46c7-ba04-dfec26175725 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:46.433438 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed771d-bd2c-46c7-ba04-dfec26175725 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:46.433551 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed771d-bd2c-46c7-ba04-dfec26175725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:46.451928 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-b4ed771d-bd2c-46c7-ba04-dfec26175725 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:46.453139 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-b4ed771d-bd2c-46c7-ba04-dfec26175725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:46.461215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 679/1387] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:37:46.507739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db7133-0b1d-437d-9b5e-fbe29b05b125 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:37:46.525839 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db7133-0b1d-437d-9b5e-fbe29b05b125 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:37:46.533112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3203e42-5076-464c-8a63-4acb57277f55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:37:46.534109 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3203e42-5076-464c-8a63-4acb57277f55 service nova] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:46.534492 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3203e42-5076-464c-8a63-4acb57277f55 service nova] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:46.535142 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3203e42-5076-464c-8a63-4acb57277f55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:46.545990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3203e42-5076-464c-8a63-4acb57277f55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:37:46.548266 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3203e42-5076-464c-8a63-4acb57277f55 service nova] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:37:46.561685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 709/1388] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:37:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:37:48.735673 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcadaa-c4cc-4a4f-a8af-f840ed006313 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:48.736764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcadaa-c4cc-4a4f-a8af-f840ed006313 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:48.736856 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcadaa-c4cc-4a4f-a8af-f840ed006313 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:48.738228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 680/1389] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:37:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:48.743875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a813cd-1cdc-4ba7-84b8-f4847e2a21ac None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:48.744314 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a813cd-1cdc-4ba7-84b8-f4847e2a21ac None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:48.744411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a813cd-1cdc-4ba7-84b8-f4847e2a21ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:48.764887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-e8a813cd-1cdc-4ba7-84b8-f4847e2a21ac None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:48.766083 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-e8a813cd-1cdc-4ba7-84b8-f4847e2a21ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:48.774116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 710/1390] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:37:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:37:48.820595 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcadaa-c4cc-4a4f-a8af-f840ed006313 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:37:48.835634 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcadaa-c4cc-4a4f-a8af-f840ed006313 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:37:48.840922 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965c2d4-d865-42d0-83bb-8cd8fe0d3f1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:37:48.841555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965c2d4-d865-42d0-83bb-8cd8fe0d3f1d service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:48.841832 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965c2d4-d865-42d0-83bb-8cd8fe0d3f1d service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:48.842103 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965c2d4-d865-42d0-83bb-8cd8fe0d3f1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:48.851464 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965c2d4-d865-42d0-83bb-8cd8fe0d3f1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:37:48.853073 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965c2d4-d865-42d0-83bb-8cd8fe0d3f1d service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:37:48.864701 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 681/1391] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:37:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:37:51.113994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ae495-dda3-4d8f-821f-c515f1b8de2d None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:51.114589 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ae495-dda3-4d8f-821f-c515f1b8de2d None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:51.114589 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ae495-dda3-4d8f-821f-c515f1b8de2d None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:51.115957 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 711/1392] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:51.121717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bda0f0-ef1e-4d3d-960c-eaea1e5f5073 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:51.122113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bda0f0-ef1e-4d3d-960c-eaea1e5f5073 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:51.122507 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bda0f0-ef1e-4d3d-960c-eaea1e5f5073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:51.146193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-21bda0f0-ef1e-4d3d-960c-eaea1e5f5073 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:51.147733 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-21bda0f0-ef1e-4d3d-960c-eaea1e5f5073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:51.158324 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 682/1393] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:37:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:37:51.202799 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ae495-dda3-4d8f-821f-c515f1b8de2d None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:37:51.217518 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ae495-dda3-4d8f-821f-c515f1b8de2d None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:37:51.224029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243fb88-94f8-46c8-a2aa-1b5b394e54f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:37:51.224661 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243fb88-94f8-46c8-a2aa-1b5b394e54f5 service nova] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:51.224944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243fb88-94f8-46c8-a2aa-1b5b394e54f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:51.225214 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243fb88-94f8-46c8-a2aa-1b5b394e54f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:51.233866 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243fb88-94f8-46c8-a2aa-1b5b394e54f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:37:51.235460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243fb88-94f8-46c8-a2aa-1b5b394e54f5 service nova] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:37:51.246400 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 712/1394] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:37:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:37:53.486217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b00d0-bf03-4175-bb4b-69185a0b1dee None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:53.486855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b00d0-bf03-4175-bb4b-69185a0b1dee None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:53.486855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b00d0-bf03-4175-bb4b-69185a0b1dee None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:53.488123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 683/1395] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:37:53.493547 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c76741-465f-463d-8b1f-6c60d28db251 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:53.493971 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c76741-465f-463d-8b1f-6c60d28db251 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:53.494096 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c76741-465f-463d-8b1f-6c60d28db251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:53.512191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-26c76741-465f-463d-8b1f-6c60d28db251 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:53.513369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-26c76741-465f-463d-8b1f-6c60d28db251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:53.521816 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 713/1396] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:37:53.566198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b00d0-bf03-4175-bb4b-69185a0b1dee None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:37:53.581548 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b00d0-bf03-4175-bb4b-69185a0b1dee None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:37:53.586808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cecc94b-aa57-4838-a947-c820eb737101 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:37:53.587454 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecc94b-aa57-4838-a947-c820eb737101 service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:53.587732 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecc94b-aa57-4838-a947-c820eb737101 service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:53.588001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecc94b-aa57-4838-a947-c820eb737101 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:53.596852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cecc94b-aa57-4838-a947-c820eb737101 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:37:53.600414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cecc94b-aa57-4838-a947-c820eb737101 service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:37:53.610032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 684/1397] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:37:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:37:53.643767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63733f1-9400-4fdd-95b9-4765b5378982 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:53.644049 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63733f1-9400-4fdd-95b9-4765b5378982 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:53.644324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63733f1-9400-4fdd-95b9-4765b5378982 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:53.645696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 714/1398] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:37:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:53.651174 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae863a-84d1-41fd-a53e-06918d01caf0 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:53.651462 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae863a-84d1-41fd-a53e-06918d01caf0 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:53.651737 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae863a-84d1-41fd-a53e-06918d01caf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:53.669132 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-26ae863a-84d1-41fd-a53e-06918d01caf0 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:53.670322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-26ae863a-84d1-41fd-a53e-06918d01caf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:53.678238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 685/1399] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:37:53.735373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035fdc0-614a-459f-a904-42f87ba4e5f8 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:53.740168 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035fdc0-614a-459f-a904-42f87ba4e5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:53.740168 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035fdc0-614a-459f-a904-42f87ba4e5f8 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:53.740168 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 715/1400] 10.222.0.15 () {54 vars in 962 bytes} [Mon Jun 27 18:37:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:53.743381 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6715d4-7612-461e-ad9e-f45528bed6a7 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:53.743672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6715d4-7612-461e-ad9e-f45528bed6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:53.743956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6715d4-7612-461e-ad9e-f45528bed6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:53.792470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-2f6715d4-7612-461e-ad9e-f45528bed6a7 None None] MFA Rules not processed for user `75e75e5ae3ca45d0b503f6e52f399558`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:53.793606 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-2f6715d4-7612-461e-ad9e-f45528bed6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:53.831530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:37:53.864453 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 686/1401] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Jun 27 18:37:53 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:37:53.872236 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035fdc0-614a-459f-a904-42f87ba4e5f8 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:37:53.885210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035fdc0-614a-459f-a904-42f87ba4e5f8 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:37:53.888940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0985a6e-00c9-4f65-b170-94976bdb01f0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e75e5ae3ca45d0b503f6e52f399558', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:37:53.889617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0985a6e-00c9-4f65-b170-94976bdb01f0 service glance] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:53.889897 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0985a6e-00c9-4f65-b170-94976bdb01f0 service glance] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:53.890172 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0985a6e-00c9-4f65-b170-94976bdb01f0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:53.900252 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0985a6e-00c9-4f65-b170-94976bdb01f0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:37:53.901703 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0985a6e-00c9-4f65-b170-94976bdb01f0 service glance] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:37:53.913176 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 716/1402] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:37:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:37:54.809479 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4b783-c6f3-468e-acb2-d689fe6dd92e None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:54.809940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4b783-c6f3-468e-acb2-d689fe6dd92e None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:54.810046 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4b783-c6f3-468e-acb2-d689fe6dd92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:54.859447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-c1d4b783-c6f3-468e-acb2-d689fe6dd92e None None] MFA Rules not processed for user `58e1f1ac6ea14da0be6e9a24d007847a`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:54.860669 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-c1d4b783-c6f3-468e-acb2-d689fe6dd92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:54.928686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 687/1403] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Jun 27 18:37:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:37:54.973587 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b850f2-d843-4d41-91a9-d0ffd74afbb1 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:54.974104 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b850f2-d843-4d41-91a9-d0ffd74afbb1 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:54.974190 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b850f2-d843-4d41-91a9-d0ffd74afbb1 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:54.975579 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 717/1404] 10.222.0.15 () {54 vars in 942 bytes} [Mon Jun 27 18:37:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:54.981914 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcbeb5-1b24-4999-aa11-0586f393fd1d None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:54.982204 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcbeb5-1b24-4999-aa11-0586f393fd1d None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:54.982511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcbeb5-1b24-4999-aa11-0586f393fd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:55.029915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-85dcbeb5-1b24-4999-aa11-0586f393fd1d None None] MFA Rules not processed for user `a1529c9a45434a19968e924797688f06`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:55.031121 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-85dcbeb5-1b24-4999-aa11-0586f393fd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:55.100744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 688/1405] 10.222.0.15 () {58 vars in 1039 bytes} [Mon Jun 27 18:37:54 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:37:55.109868 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b850f2-d843-4d41-91a9-d0ffd74afbb1 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:37:55.118181 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=61 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:37:55.124464 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b850f2-d843-4d41-91a9-d0ffd74afbb1 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:37:55.129336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792e3cb-dc23-481c-908a-0e5c81f5f7f2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1529c9a45434a19968e924797688f06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:37:55.133398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792e3cb-dc23-481c-908a-0e5c81f5f7f2 service swift] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:55.133398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792e3cb-dc23-481c-908a-0e5c81f5f7f2 service swift] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:55.133398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792e3cb-dc23-481c-908a-0e5c81f5f7f2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:55.138315 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792e3cb-dc23-481c-908a-0e5c81f5f7f2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:37:55.139901 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792e3cb-dc23-481c-908a-0e5c81f5f7f2 service swift] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:37:55.148053 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 718/1406] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Jun 27 18:37:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 18:37:55.840264 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c098428-1687-447e-84f9-ef460d9640ed None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:55.840937 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c098428-1687-447e-84f9-ef460d9640ed None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:55.840937 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c098428-1687-447e-84f9-ef460d9640ed None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:55.843249 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 689/1407] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:37:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:37:55.849162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fba00-ded3-4257-bed6-e882d3736fa3 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:55.849566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fba00-ded3-4257-bed6-e882d3736fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:55.851247 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fba00-ded3-4257-bed6-e882d3736fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:55.875072 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-8e0fba00-ded3-4257-bed6-e882d3736fa3 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:37:55.877517 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-8e0fba00-ded3-4257-bed6-e882d3736fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:37:55.890805 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 719/1408] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:37:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:37:55.932044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c098428-1687-447e-84f9-ef460d9640ed None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:37:55.948736 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c098428-1687-447e-84f9-ef460d9640ed None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:37:55.956776 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44aaf54-68c4-4241-8991-d819898a20c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:37:55.957436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44aaf54-68c4-4241-8991-d819898a20c6 service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:37:55.957719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44aaf54-68c4-4241-8991-d819898a20c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:37:55.957995 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44aaf54-68c4-4241-8991-d819898a20c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:37:55.967850 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44aaf54-68c4-4241-8991-d819898a20c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:37:55.994917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44aaf54-68c4-4241-8991-d819898a20c6 service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:37:55.994917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:37:55.994917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 690/1409] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:37:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:38:05.866583 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=8 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:38:18.335310 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9d540-1a22-4cb2-91fe-ab64f4bdcfe9 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:18.336004 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9d540-1a22-4cb2-91fe-ab64f4bdcfe9 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:18.336004 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9d540-1a22-4cb2-91fe-ab64f4bdcfe9 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:18.337260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 720/1410] 10.222.0.15 () {54 vars in 918 bytes} [Mon Jun 27 18:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:38:18.342301 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0627b-1713-4a22-aafa-471549066f8a None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:18.342783 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0627b-1713-4a22-aafa-471549066f8a None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:18.342888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0627b-1713-4a22-aafa-471549066f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:18.361486 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-e9f0627b-1713-4a22-aafa-471549066f8a None None] MFA Rules not processed for user `3cd6f0a1286243e4add692db01028205`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:38:18.362699 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-e9f0627b-1713-4a22-aafa-471549066f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:38:18.371306 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 691/1411] 10.222.0.15 () {58 vars in 1015 bytes} [Mon Jun 27 18:38:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:38:18.385143 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da7e5d-f473-41cc-928c-531aafe020f1 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:18.385435 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da7e5d-f473-41cc-928c-531aafe020f1 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:18.385717 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da7e5d-f473-41cc-928c-531aafe020f1 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:18.387156 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 721/1412] 10.222.0.15 () {54 vars in 918 bytes} [Mon Jun 27 18:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:38:18.391702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1e33c-d309-42ff-ae55-6ba0fafb23ad None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:18.391981 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1e33c-d309-42ff-ae55-6ba0fafb23ad None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:18.392257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1e33c-d309-42ff-ae55-6ba0fafb23ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:18.409935 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-bdf1e33c-d309-42ff-ae55-6ba0fafb23ad None None] MFA Rules not processed for user `1dddd4efb44c4152bb4151f7715f2223`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:38:18.411142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-bdf1e33c-d309-42ff-ae55-6ba0fafb23ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:38:18.420360 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 692/1413] 10.222.0.15 () {58 vars in 1015 bytes} [Mon Jun 27 18:38:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:38:20.919504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae3be4-6cc7-4981-afab-3d7cdfd1062b None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:20.920082 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae3be4-6cc7-4981-afab-3d7cdfd1062b None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:20.920082 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae3be4-6cc7-4981-afab-3d7cdfd1062b None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:20.921424 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 722/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 18:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:38:20.927221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b11dd-db40-4f51-a33c-c4a06f3761b5 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:20.927656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b11dd-db40-4f51-a33c-c4a06f3761b5 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:20.927777 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b11dd-db40-4f51-a33c-c4a06f3761b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:20.945914 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-822b11dd-db40-4f51-a33c-c4a06f3761b5 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:38:20.947121 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-822b11dd-db40-4f51-a33c-c4a06f3761b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:38:20.955135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 693/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 18:38:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:38:20.998884 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae3be4-6cc7-4981-afab-3d7cdfd1062b None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:38:21.013554 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae3be4-6cc7-4981-afab-3d7cdfd1062b None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:38:21.019137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14883f95-9cc9-43a2-a95d-faccc117e51f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:38:21.019760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14883f95-9cc9-43a2-a95d-faccc117e51f service nova] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:21.020044 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14883f95-9cc9-43a2-a95d-faccc117e51f service nova] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:21.020320 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14883f95-9cc9-43a2-a95d-faccc117e51f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:21.028306 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14883f95-9cc9-43a2-a95d-faccc117e51f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:38:21.029873 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14883f95-9cc9-43a2-a95d-faccc117e51f service nova] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:38:21.039230 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 723/1416] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:38:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:38:21.936619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a8e02-0af6-4901-a966-1bdb7cde7278 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:21.937158 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a8e02-0af6-4901-a966-1bdb7cde7278 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:21.937158 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a8e02-0af6-4901-a966-1bdb7cde7278 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:21.939099 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 694/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 18:38:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:38:21.943894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbddbf-fc87-4fea-bf15-af112fe12983 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:21.944327 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbddbf-fc87-4fea-bf15-af112fe12983 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:21.944445 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbddbf-fc87-4fea-bf15-af112fe12983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:21.962835 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-fdbbddbf-fc87-4fea-bf15-af112fe12983 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:38:21.964019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-fdbbddbf-fc87-4fea-bf15-af112fe12983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:38:21.971958 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 724/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 18:38:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:38:22.122690 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a8e02-0af6-4901-a966-1bdb7cde7278 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:38:22.137326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a8e02-0af6-4901-a966-1bdb7cde7278 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:38:22.143585 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2545cfb-bd47-4639-8342-21844479ebe0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:38:22.144219 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2545cfb-bd47-4639-8342-21844479ebe0 service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:22.144500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2545cfb-bd47-4639-8342-21844479ebe0 service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:22.144776 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2545cfb-bd47-4639-8342-21844479ebe0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:22.153012 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2545cfb-bd47-4639-8342-21844479ebe0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:38:22.154619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2545cfb-bd47-4639-8342-21844479ebe0 service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:38:22.163992 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 695/1419] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:38:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:38:23.041388 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea6f8b-f48c-4fe2-b418-ddc27f8bbf81 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:23.041930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea6f8b-f48c-4fe2-b418-ddc27f8bbf81 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:23.041930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea6f8b-f48c-4fe2-b418-ddc27f8bbf81 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:23.043280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 725/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 18:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:38:23.048376 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79de12-497a-471b-b334-7463b6ce59ad None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:23.048783 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79de12-497a-471b-b334-7463b6ce59ad None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:23.048943 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79de12-497a-471b-b334-7463b6ce59ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:23.066537 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-6b79de12-497a-471b-b334-7463b6ce59ad None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:38:23.067745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-6b79de12-497a-471b-b334-7463b6ce59ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:38:23.075450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 696/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 18:38:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:38:23.227077 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea6f8b-f48c-4fe2-b418-ddc27f8bbf81 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:38:23.241917 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea6f8b-f48c-4fe2-b418-ddc27f8bbf81 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:38:23.247242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894cb13-e0b2-4879-96b9-3be53e4f0ddb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:38:23.247866 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894cb13-e0b2-4879-96b9-3be53e4f0ddb service nova] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:23.248149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894cb13-e0b2-4879-96b9-3be53e4f0ddb service nova] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:23.248426 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894cb13-e0b2-4879-96b9-3be53e4f0ddb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:23.257064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f894cb13-e0b2-4879-96b9-3be53e4f0ddb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:38:23.258656 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f894cb13-e0b2-4879-96b9-3be53e4f0ddb service nova] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:38:23.268137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 726/1422] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:38:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:38:24.121966 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75260e44-eec6-49b4-bf9d-95f5d7f960b1 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:24.122578 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75260e44-eec6-49b4-bf9d-95f5d7f960b1 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:24.122712 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75260e44-eec6-49b4-bf9d-95f5d7f960b1 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:24.123992 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 697/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 18:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:38:24.129344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9357744-dafc-4608-8950-c0c7b57acd82 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:24.129743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9357744-dafc-4608-8950-c0c7b57acd82 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:24.129893 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9357744-dafc-4608-8950-c0c7b57acd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:24.148420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-e9357744-dafc-4608-8950-c0c7b57acd82 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:38:24.149622 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-e9357744-dafc-4608-8950-c0c7b57acd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:38:24.158100 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 727/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 18:38:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:38:24.309867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75260e44-eec6-49b4-bf9d-95f5d7f960b1 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:38:24.324696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75260e44-eec6-49b4-bf9d-95f5d7f960b1 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:38:24.331294 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737ffb5-d1fb-4059-b3d4-0fe13debd48a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:38:24.331923 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737ffb5-d1fb-4059-b3d4-0fe13debd48a service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:24.332200 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737ffb5-d1fb-4059-b3d4-0fe13debd48a service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:24.332473 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737ffb5-d1fb-4059-b3d4-0fe13debd48a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:24.340627 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737ffb5-d1fb-4059-b3d4-0fe13debd48a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:38:24.342193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737ffb5-d1fb-4059-b3d4-0fe13debd48a service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:38:24.351513 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 698/1425] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:38:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:38:25.207614 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779f59d-371d-4434-b837-a7e5f8daf180 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:25.208187 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779f59d-371d-4434-b837-a7e5f8daf180 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:25.208187 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779f59d-371d-4434-b837-a7e5f8daf180 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:25.209584 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 728/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 18:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:38:25.214505 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e026e-0420-42b8-90d6-2fac1a2ab1ce None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:25.214943 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e026e-0420-42b8-90d6-2fac1a2ab1ce None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:25.215051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e026e-0420-42b8-90d6-2fac1a2ab1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:25.232949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-3d8e026e-0420-42b8-90d6-2fac1a2ab1ce None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:38:25.234125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-3d8e026e-0420-42b8-90d6-2fac1a2ab1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:38:25.242754 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 699/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 18:38:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:38:25.394334 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779f59d-371d-4434-b837-a7e5f8daf180 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:38:25.409139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779f59d-371d-4434-b837-a7e5f8daf180 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:38:25.414464 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e38bf-6316-430b-8a8a-1cef464b3b6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:38:25.415087 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e38bf-6316-430b-8a8a-1cef464b3b6f service nova] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:25.415371 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e38bf-6316-430b-8a8a-1cef464b3b6f service nova] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:25.415645 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e38bf-6316-430b-8a8a-1cef464b3b6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:25.424336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e38bf-6316-430b-8a8a-1cef464b3b6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:38:25.425926 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e38bf-6316-430b-8a8a-1cef464b3b6f service nova] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:38:25.435488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 729/1428] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:38:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:38:26.327297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4efae5-c229-4258-bb57-dc2b7864c05f None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:26.327827 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4efae5-c229-4258-bb57-dc2b7864c05f None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:26.327827 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4efae5-c229-4258-bb57-dc2b7864c05f None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:26.329942 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 700/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 18:38:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:38:26.334777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941157dc-5f7b-4cd2-8610-d44b3a4e04a7 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:26.335209 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941157dc-5f7b-4cd2-8610-d44b3a4e04a7 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:26.335310 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941157dc-5f7b-4cd2-8610-d44b3a4e04a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:26.353308 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-941157dc-5f7b-4cd2-8610-d44b3a4e04a7 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:38:26.354544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-941157dc-5f7b-4cd2-8610-d44b3a4e04a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:38:26.362927 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 730/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 18:38:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:38:26.514222 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4efae5-c229-4258-bb57-dc2b7864c05f None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:38:26.530105 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4efae5-c229-4258-bb57-dc2b7864c05f None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:38:26.535881 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ceaa6d-4302-4e1e-8542-f193a2ee3671 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:38:26.536499 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ceaa6d-4302-4e1e-8542-f193a2ee3671 service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:26.536779 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ceaa6d-4302-4e1e-8542-f193a2ee3671 service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:26.537049 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ceaa6d-4302-4e1e-8542-f193a2ee3671 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:26.544966 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ceaa6d-4302-4e1e-8542-f193a2ee3671 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:38:26.546552 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ceaa6d-4302-4e1e-8542-f193a2ee3671 service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:38:26.556120 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 701/1431] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:38:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:38:27.417675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30dd5f-e909-43e0-b1df-1f1650861ee9 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:27.418222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30dd5f-e909-43e0-b1df-1f1650861ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:27.418222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30dd5f-e909-43e0-b1df-1f1650861ee9 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:27.419634 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 731/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 18:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:38:27.425152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39139992-ced8-4978-a9e4-b76afe92b590 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:27.425684 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39139992-ced8-4978-a9e4-b76afe92b590 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:27.425684 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39139992-ced8-4978-a9e4-b76afe92b590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:27.443187 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-39139992-ced8-4978-a9e4-b76afe92b590 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:38:27.444374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-39139992-ced8-4978-a9e4-b76afe92b590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:38:27.452395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 702/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 18:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:38:27.603061 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30dd5f-e909-43e0-b1df-1f1650861ee9 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:38:27.617899 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30dd5f-e909-43e0-b1df-1f1650861ee9 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:38:27.623132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a655bca1-4521-4372-b404-3330741350b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:38:27.623748 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655bca1-4521-4372-b404-3330741350b7 service nova] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:27.624025 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655bca1-4521-4372-b404-3330741350b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:27.624294 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655bca1-4521-4372-b404-3330741350b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:27.633583 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a655bca1-4521-4372-b404-3330741350b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:38:27.635562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a655bca1-4521-4372-b404-3330741350b7 service nova] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:38:27.644675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 732/1434] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:38:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:38:28.508447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4aac5c-df5d-4e4b-b12f-56919c4abf0d None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:28.509015 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4aac5c-df5d-4e4b-b12f-56919c4abf0d None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:28.509015 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4aac5c-df5d-4e4b-b12f-56919c4abf0d None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:28.510296 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 703/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 18:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:38:28.520383 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d04155-50af-41e5-9cb1-7dfc8b6e912a None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:28.520806 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d04155-50af-41e5-9cb1-7dfc8b6e912a None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:28.520915 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d04155-50af-41e5-9cb1-7dfc8b6e912a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:28.539526 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-85d04155-50af-41e5-9cb1-7dfc8b6e912a None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:38:28.540710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-85d04155-50af-41e5-9cb1-7dfc8b6e912a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:38:28.548555 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 733/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 18:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:38:28.706795 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4aac5c-df5d-4e4b-b12f-56919c4abf0d None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:38:28.721141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4aac5c-df5d-4e4b-b12f-56919c4abf0d None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:38:28.726374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a1e07-86b0-400b-bdcb-ab157ae5ac58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:38:28.727020 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a1e07-86b0-400b-bdcb-ab157ae5ac58 service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:28.727302 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a1e07-86b0-400b-bdcb-ab157ae5ac58 service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:28.727573 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a1e07-86b0-400b-bdcb-ab157ae5ac58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:28.735811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a1e07-86b0-400b-bdcb-ab157ae5ac58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:38:28.737385 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a1e07-86b0-400b-bdcb-ab157ae5ac58 service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:38:28.747318 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 704/1437] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:38:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:38:29.613050 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494842c-6d98-489e-beaa-9dd201f5bb30 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:29.613884 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494842c-6d98-489e-beaa-9dd201f5bb30 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:29.613884 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494842c-6d98-489e-beaa-9dd201f5bb30 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:29.615260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 734/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 18:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:38:29.620326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc95138-695b-4f08-acf4-7c5fb596d93c None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:29.620721 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc95138-695b-4f08-acf4-7c5fb596d93c None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:29.620875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc95138-695b-4f08-acf4-7c5fb596d93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:29.639332 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-8bc95138-695b-4f08-acf4-7c5fb596d93c None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:38:29.640515 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-8bc95138-695b-4f08-acf4-7c5fb596d93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:38:29.649458 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 705/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 18:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:38:29.804470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494842c-6d98-489e-beaa-9dd201f5bb30 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:38:29.819777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494842c-6d98-489e-beaa-9dd201f5bb30 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:38:29.825186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc95110-2ece-4798-8615-ed7d615a2990 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:38:29.825814 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc95110-2ece-4798-8615-ed7d615a2990 service nova] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:29.826094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc95110-2ece-4798-8615-ed7d615a2990 service nova] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:29.826367 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc95110-2ece-4798-8615-ed7d615a2990 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:29.834892 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc95110-2ece-4798-8615-ed7d615a2990 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:38:29.840734 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc95110-2ece-4798-8615-ed7d615a2990 service nova] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:38:29.846431 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 735/1440] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:38:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:38:31.786487 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c5c01-4f50-41ae-8957-ca371cc5361e None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:31.787033 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c5c01-4f50-41ae-8957-ca371cc5361e None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:31.787033 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c5c01-4f50-41ae-8957-ca371cc5361e None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:31.788426 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 706/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 18:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:38:31.793523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769cf6fd-5271-4121-bdb6-c9ab8d967cee None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:31.793931 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769cf6fd-5271-4121-bdb6-c9ab8d967cee None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:31.794073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769cf6fd-5271-4121-bdb6-c9ab8d967cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:31.812293 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-769cf6fd-5271-4121-bdb6-c9ab8d967cee None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:38:31.813706 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-769cf6fd-5271-4121-bdb6-c9ab8d967cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:38:31.821428 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 736/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 18:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:38:31.860275 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c5c01-4f50-41ae-8957-ca371cc5361e None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:38:31.871051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c5c01-4f50-41ae-8957-ca371cc5361e None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:38:31.874476 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3826a9-e898-4980-ba5a-40eceb601f6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:38:31.875185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3826a9-e898-4980-ba5a-40eceb601f6a service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:31.875185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3826a9-e898-4980-ba5a-40eceb601f6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:31.875721 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3826a9-e898-4980-ba5a-40eceb601f6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:31.884346 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3826a9-e898-4980-ba5a-40eceb601f6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:38:31.885754 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3826a9-e898-4980-ba5a-40eceb601f6a service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:38:31.895233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 707/1443] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:38:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:38:34.088965 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08867be1-e351-49ee-9dba-f027db7ea81f None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:34.089517 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08867be1-e351-49ee-9dba-f027db7ea81f None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:34.089517 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08867be1-e351-49ee-9dba-f027db7ea81f None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:34.090973 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 737/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 18:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:38:34.096668 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e3d61-426e-4668-88f4-a40ccf0abf48 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:34.097082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e3d61-426e-4668-88f4-a40ccf0abf48 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:34.097249 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e3d61-426e-4668-88f4-a40ccf0abf48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:34.115191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-539e3d61-426e-4668-88f4-a40ccf0abf48 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:38:34.116382 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-539e3d61-426e-4668-88f4-a40ccf0abf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:38:34.124103 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 708/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 18:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:38:34.162625 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08867be1-e351-49ee-9dba-f027db7ea81f None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:38:34.173385 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08867be1-e351-49ee-9dba-f027db7ea81f None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:38:34.176909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e191c-d8d9-4840-ad86-aa35853840c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:38:34.177531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e191c-d8d9-4840-ad86-aa35853840c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:34.177815 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e191c-d8d9-4840-ad86-aa35853840c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:34.178095 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e191c-d8d9-4840-ad86-aa35853840c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:34.186868 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e191c-d8d9-4840-ad86-aa35853840c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:38:34.188888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e191c-d8d9-4840-ad86-aa35853840c2 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:38:34.198719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 738/1446] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:38:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:38:36.580424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580323d-ab44-4b14-8558-d71e6d39829d None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:36.581044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580323d-ab44-4b14-8558-d71e6d39829d None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:36.581044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580323d-ab44-4b14-8558-d71e6d39829d None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:36.582342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 709/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 18:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:38:36.587770 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c4fa4-bd1b-4980-8871-cf79595a2f19 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:36.588193 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c4fa4-bd1b-4980-8871-cf79595a2f19 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:36.588342 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c4fa4-bd1b-4980-8871-cf79595a2f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:36.606797 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-ec8c4fa4-bd1b-4980-8871-cf79595a2f19 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:38:36.607979 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ec8c4fa4-bd1b-4980-8871-cf79595a2f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:38:36.616139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 739/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 18:38:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:38:36.654398 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580323d-ab44-4b14-8558-d71e6d39829d None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:38:36.666581 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580323d-ab44-4b14-8558-d71e6d39829d None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:38:36.670130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb5a929-c9fb-40c9-afd4-f884b42c4671 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:38:36.670790 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb5a929-c9fb-40c9-afd4-f884b42c4671 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:36.672510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb5a929-c9fb-40c9-afd4-f884b42c4671 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:36.672510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb5a929-c9fb-40c9-afd4-f884b42c4671 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:36.680442 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb5a929-c9fb-40c9-afd4-f884b42c4671 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:38:36.681860 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb5a929-c9fb-40c9-afd4-f884b42c4671 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:38:36.691135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 710/1449] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:38:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:38:38.849833 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df8740-bd30-4956-966b-6dc15c1f48e9 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:38.850370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df8740-bd30-4956-966b-6dc15c1f48e9 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:38.850370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df8740-bd30-4956-966b-6dc15c1f48e9 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:38.851784 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 740/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 18:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:38:38.857243 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b71fa6-55a0-4ef8-a212-0a87dce43cc1 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:38.857694 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b71fa6-55a0-4ef8-a212-0a87dce43cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:38.857803 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b71fa6-55a0-4ef8-a212-0a87dce43cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:38.875503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-e5b71fa6-55a0-4ef8-a212-0a87dce43cc1 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:38:38.876695 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-e5b71fa6-55a0-4ef8-a212-0a87dce43cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:38:38.884401 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 711/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 18:38:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:38:38.923245 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df8740-bd30-4956-966b-6dc15c1f48e9 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:38:38.933679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df8740-bd30-4956-966b-6dc15c1f48e9 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:38:38.937118 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50253b19-b42b-41d5-9131-d135d1a36102 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:38:38.937819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50253b19-b42b-41d5-9131-d135d1a36102 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:38.938120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50253b19-b42b-41d5-9131-d135d1a36102 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:38.938434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50253b19-b42b-41d5-9131-d135d1a36102 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:38.947121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50253b19-b42b-41d5-9131-d135d1a36102 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:38:38.948524 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50253b19-b42b-41d5-9131-d135d1a36102 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:38:38.958286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 741/1452] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:38:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:38:41.059582 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31827c-d427-44d7-8e99-4a0dd07e1985 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:41.060109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31827c-d427-44d7-8e99-4a0dd07e1985 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:41.060109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31827c-d427-44d7-8e99-4a0dd07e1985 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:41.061429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 712/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 18:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:38:41.066820 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a006a-90d8-4775-af64-3aa2aaa96c94 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:41.067212 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a006a-90d8-4775-af64-3aa2aaa96c94 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:41.067406 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a006a-90d8-4775-af64-3aa2aaa96c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:41.085119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-5d7a006a-90d8-4775-af64-3aa2aaa96c94 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:38:41.086309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-5d7a006a-90d8-4775-af64-3aa2aaa96c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:38:41.094514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 742/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 18:38:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:38:41.132023 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31827c-d427-44d7-8e99-4a0dd07e1985 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:38:41.144116 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31827c-d427-44d7-8e99-4a0dd07e1985 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:38:41.147601 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0fb1ff-d13c-4ba7-a567-7474f59288b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:38:41.148253 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fb1ff-d13c-4ba7-a567-7474f59288b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:41.148533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fb1ff-d13c-4ba7-a567-7474f59288b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:41.148812 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fb1ff-d13c-4ba7-a567-7474f59288b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:41.158688 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0fb1ff-d13c-4ba7-a567-7474f59288b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:38:41.160106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0fb1ff-d13c-4ba7-a567-7474f59288b6 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:38:41.169868 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 713/1455] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:38:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:38:43.387537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c9f3d-3303-47ea-9c9c-93b1917c1d19 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:43.388108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c9f3d-3303-47ea-9c9c-93b1917c1d19 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:43.388108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c9f3d-3303-47ea-9c9c-93b1917c1d19 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:43.389477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 743/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 18:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:38:43.395644 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b775c-5a2c-47bd-9f8a-171b05a10922 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:43.396039 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b775c-5a2c-47bd-9f8a-171b05a10922 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:43.396191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b775c-5a2c-47bd-9f8a-171b05a10922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:43.414279 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-301b775c-5a2c-47bd-9f8a-171b05a10922 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:38:43.415485 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-301b775c-5a2c-47bd-9f8a-171b05a10922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:38:43.423570 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 714/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 18:38:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:38:43.462224 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c9f3d-3303-47ea-9c9c-93b1917c1d19 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:38:43.473534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c9f3d-3303-47ea-9c9c-93b1917c1d19 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:38:43.477045 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552d9db-dda4-43cc-9121-4fd2d2027615 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:38:43.477663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552d9db-dda4-43cc-9121-4fd2d2027615 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:43.477946 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552d9db-dda4-43cc-9121-4fd2d2027615 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:43.478219 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552d9db-dda4-43cc-9121-4fd2d2027615 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:43.487143 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552d9db-dda4-43cc-9121-4fd2d2027615 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:38:43.488818 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552d9db-dda4-43cc-9121-4fd2d2027615 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:38:43.498038 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 744/1458] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:38:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:38:45.692540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdb659-496c-4971-a990-f4287cbc1224 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:45.693118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdb659-496c-4971-a990-f4287cbc1224 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:45.693118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdb659-496c-4971-a990-f4287cbc1224 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:45.694505 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 715/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 18:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:38:45.699620 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3c1cb-bb78-43de-b56d-5a6f4cf5ac2e None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:45.700022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3c1cb-bb78-43de-b56d-5a6f4cf5ac2e None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:45.700189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3c1cb-bb78-43de-b56d-5a6f4cf5ac2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:45.717868 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-96e3c1cb-bb78-43de-b56d-5a6f4cf5ac2e None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:38:45.719712 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-96e3c1cb-bb78-43de-b56d-5a6f4cf5ac2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:38:45.727416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 745/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 18:38:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:38:45.764461 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdb659-496c-4971-a990-f4287cbc1224 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:38:45.776033 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdb659-496c-4971-a990-f4287cbc1224 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:38:45.779482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b909743-edc8-4fa7-875e-598d354a3f5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:38:45.780110 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b909743-edc8-4fa7-875e-598d354a3f5a service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:45.780390 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b909743-edc8-4fa7-875e-598d354a3f5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:45.780713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b909743-edc8-4fa7-875e-598d354a3f5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:45.789973 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b909743-edc8-4fa7-875e-598d354a3f5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:38:45.791415 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b909743-edc8-4fa7-875e-598d354a3f5a service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:38:45.800659 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 716/1461] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:38:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:38:48.017077 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f7d19-69da-4cca-b792-e55addc97dcd None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:48.017612 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f7d19-69da-4cca-b792-e55addc97dcd None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:48.017612 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f7d19-69da-4cca-b792-e55addc97dcd None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:48.018938 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 746/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 18:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:38:48.024367 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22060e-6622-4154-9c01-22db42a2cf68 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:48.024788 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22060e-6622-4154-9c01-22db42a2cf68 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:48.024932 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22060e-6622-4154-9c01-22db42a2cf68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:48.031232 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=79 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:38:48.043506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-be22060e-6622-4154-9c01-22db42a2cf68 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:38:48.044703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-be22060e-6622-4154-9c01-22db42a2cf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:38:48.053078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 717/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 18:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:38:48.090081 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f7d19-69da-4cca-b792-e55addc97dcd None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:38:48.095592 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=73 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:38:48.101090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f7d19-69da-4cca-b792-e55addc97dcd None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:38:48.104561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e5e28-5ca5-40ea-a513-eebc5708ec9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:38:48.105230 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e5e28-5ca5-40ea-a513-eebc5708ec9c service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:48.105510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e5e28-5ca5-40ea-a513-eebc5708ec9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:48.105781 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e5e28-5ca5-40ea-a513-eebc5708ec9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:48.115604 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e5e28-5ca5-40ea-a513-eebc5708ec9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:38:48.117052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e5e28-5ca5-40ea-a513-eebc5708ec9c service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:38:48.127491 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 747/1464] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:38:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:38:50.223120 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e8d1b-0c6d-47b3-b849-a1b55e3500ed None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:50.223566 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e8d1b-0c6d-47b3-b849-a1b55e3500ed None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:50.223677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e8d1b-0c6d-47b3-b849-a1b55e3500ed None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:50.225111 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 718/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 18:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:38:50.230293 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff727ce-ef31-4af0-8848-b88c91b5506b None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:50.230730 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff727ce-ef31-4af0-8848-b88c91b5506b None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:50.230890 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff727ce-ef31-4af0-8848-b88c91b5506b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:50.250610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-dff727ce-ef31-4af0-8848-b88c91b5506b None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:38:50.251816 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-dff727ce-ef31-4af0-8848-b88c91b5506b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:38:50.259589 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 748/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 18:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:38:50.295322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e8d1b-0c6d-47b3-b849-a1b55e3500ed None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:38:50.306906 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e8d1b-0c6d-47b3-b849-a1b55e3500ed None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:38:50.310332 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24e242-8c5c-4448-b670-e3345ba513e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:38:50.310994 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24e242-8c5c-4448-b670-e3345ba513e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:50.311276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24e242-8c5c-4448-b670-e3345ba513e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:50.311550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24e242-8c5c-4448-b670-e3345ba513e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:50.321155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24e242-8c5c-4448-b670-e3345ba513e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:38:50.322591 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24e242-8c5c-4448-b670-e3345ba513e9 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:38:50.331841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 719/1467] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:38:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:38:52.425323 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ac383-467c-4fed-9134-6882e3d00eca None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:52.425854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ac383-467c-4fed-9134-6882e3d00eca None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:52.425975 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ac383-467c-4fed-9134-6882e3d00eca None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:52.427328 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 749/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 18:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:38:52.432692 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f25e6-8abd-43ca-be74-1fa50cfc405c None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:52.433121 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f25e6-8abd-43ca-be74-1fa50cfc405c None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:52.433239 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f25e6-8abd-43ca-be74-1fa50cfc405c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:52.451014 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-7b3f25e6-8abd-43ca-be74-1fa50cfc405c None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:38:52.452206 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-7b3f25e6-8abd-43ca-be74-1fa50cfc405c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:38:52.460054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 720/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 18:38:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:38:52.496331 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ac383-467c-4fed-9134-6882e3d00eca None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:38:52.507602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ac383-467c-4fed-9134-6882e3d00eca None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:38:52.511091 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c7d33d-7427-43f3-8a1c-a01d167822c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:38:52.511706 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7d33d-7427-43f3-8a1c-a01d167822c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:38:52.511987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7d33d-7427-43f3-8a1c-a01d167822c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:38:52.512258 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c7d33d-7427-43f3-8a1c-a01d167822c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:38:52.520958 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c7d33d-7427-43f3-8a1c-a01d167822c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:38:52.522370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c7d33d-7427-43f3-8a1c-a01d167822c0 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:38:52.532000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 750/1470] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:38:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:39:02.446034 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=11 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:39:02.527495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=14 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:39:12.420469 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef9550-764f-4165-aa17-0ec46612c975 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:39:12.421574 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef9550-764f-4165-aa17-0ec46612c975 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:39:12.421574 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef9550-764f-4165-aa17-0ec46612c975 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:39:12.422515 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 721/1471] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:39:12.427834 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dc58d-559f-41c7-8f71-d79d25d3cc2f None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:39:12.428287 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dc58d-559f-41c7-8f71-d79d25d3cc2f None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:39:12.428478 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8dc58d-559f-41c7-8f71-d79d25d3cc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:39:12.446506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-2b8dc58d-559f-41c7-8f71-d79d25d3cc2f None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:39:12.447705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-2b8dc58d-559f-41c7-8f71-d79d25d3cc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:39:12.456511 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 751/1472] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:39:12.504833 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b04a7-bf45-46e1-8c6e-a192b286b081 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:39:12.505123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b04a7-bf45-46e1-8c6e-a192b286b081 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:39:12.505404 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b04a7-bf45-46e1-8c6e-a192b286b081 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:39:12.506804 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 722/1473] 10.222.0.15 () {54 vars in 962 bytes} [Mon Jun 27 18:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:39:12.511545 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c53fd-e878-481c-861d-12cc1529908b None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:39:12.511831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c53fd-e878-481c-861d-12cc1529908b None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:39:12.512107 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c53fd-e878-481c-861d-12cc1529908b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:39:12.531810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-5e3c53fd-e878-481c-861d-12cc1529908b None None] MFA Rules not processed for user `75e75e5ae3ca45d0b503f6e52f399558`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:39:12.532986 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-5e3c53fd-e878-481c-861d-12cc1529908b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:39:12.539306 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 752/1474] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Jun 27 18:39:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:39:12.547491 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b04a7-bf45-46e1-8c6e-a192b286b081 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:39:12.557906 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b04a7-bf45-46e1-8c6e-a192b286b081 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:39:12.561924 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310fc6c2-0b1f-49e7-85a4-b8d0310db40c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e75e5ae3ca45d0b503f6e52f399558', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:39:12.562576 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fc6c2-0b1f-49e7-85a4-b8d0310db40c service glance] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:39:12.562862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fc6c2-0b1f-49e7-85a4-b8d0310db40c service glance] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:39:12.563137 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fc6c2-0b1f-49e7-85a4-b8d0310db40c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:39:12.571970 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310fc6c2-0b1f-49e7-85a4-b8d0310db40c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:39:12.573425 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310fc6c2-0b1f-49e7-85a4-b8d0310db40c service glance] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:39:12.583034 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 723/1475] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:39:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:39:14.728814 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b12ae-2770-47c7-9963-98502e64af0e None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:39:14.729382 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b12ae-2770-47c7-9963-98502e64af0e None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:39:14.729382 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b12ae-2770-47c7-9963-98502e64af0e None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:39:14.730868 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 753/1476] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:39:14.736916 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54f7d9-d149-4d41-b763-6eacb367bbf5 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:39:14.737426 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54f7d9-d149-4d41-b763-6eacb367bbf5 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:39:14.737534 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54f7d9-d149-4d41-b763-6eacb367bbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:39:14.755637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-dd54f7d9-d149-4d41-b763-6eacb367bbf5 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:39:14.756898 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-dd54f7d9-d149-4d41-b763-6eacb367bbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:39:14.765386 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 724/1477] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:39:14.789863 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b12ae-2770-47c7-9963-98502e64af0e None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:39:14.802857 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b12ae-2770-47c7-9963-98502e64af0e None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:39:14.807301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc2126-c227-4f44-9165-f1e2f9277d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:39:14.807940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc2126-c227-4f44-9165-f1e2f9277d6b admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:39:14.808223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc2126-c227-4f44-9165-f1e2f9277d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:39:14.808499 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc2126-c227-4f44-9165-f1e2f9277d6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:39:14.809232 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc2126-c227-4f44-9165-f1e2f9277d6b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:39:14.811203 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: WARNING py.warnings [None req-0adc2126-c227-4f44-9165-f1e2f9277d6b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 18:39:14.811203 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: warnings.warn(msg) Jun 27 18:39:14.811203 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: Jun 27 18:39:14.812340 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc2126-c227-4f44-9165-f1e2f9277d6b admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:39:14.831820 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 754/1478] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Jun 27 18:39:14 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:39:16.895047 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac48202-636c-4b5b-83f7-41c2b80ef17c None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:39:16.895613 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac48202-636c-4b5b-83f7-41c2b80ef17c None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:39:16.895613 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac48202-636c-4b5b-83f7-41c2b80ef17c None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:39:16.897013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 725/1479] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:39:16.902294 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e79aa-e2d7-462e-a7a9-c98a559b533c None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:39:16.902774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e79aa-e2d7-462e-a7a9-c98a559b533c None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:39:16.902872 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e79aa-e2d7-462e-a7a9-c98a559b533c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:39:16.921454 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-213e79aa-e2d7-462e-a7a9-c98a559b533c None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:39:16.922691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-213e79aa-e2d7-462e-a7a9-c98a559b533c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:39:16.930599 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 755/1480] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:39:16.973619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac48202-636c-4b5b-83f7-41c2b80ef17c None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:39:16.990255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac48202-636c-4b5b-83f7-41c2b80ef17c None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:39:16.995747 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2fd0ab-e9f3-4643-9d46-6cc20cea67b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:39:16.996382 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fd0ab-e9f3-4643-9d46-6cc20cea67b0 service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:39:16.996667 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fd0ab-e9f3-4643-9d46-6cc20cea67b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:39:16.996990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fd0ab-e9f3-4643-9d46-6cc20cea67b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:39:17.005857 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2fd0ab-e9f3-4643-9d46-6cc20cea67b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:39:17.007614 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2fd0ab-e9f3-4643-9d46-6cc20cea67b0 service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:39:17.018140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 726/1481] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:39:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:39:19.222183 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c406386-9fe0-4f3b-b65c-39e706bd53c7 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:39:19.222823 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c406386-9fe0-4f3b-b65c-39e706bd53c7 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:39:19.222823 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c406386-9fe0-4f3b-b65c-39e706bd53c7 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:39:19.224148 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 756/1482] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:39:19.229295 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3409209-7505-4855-b189-60cbac0df12c None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:39:19.240409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3409209-7505-4855-b189-60cbac0df12c None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:39:19.240409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3409209-7505-4855-b189-60cbac0df12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:39:19.249587 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-f3409209-7505-4855-b189-60cbac0df12c None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:39:19.250818 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f3409209-7505-4855-b189-60cbac0df12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:39:19.258579 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 727/1483] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:39:19.295602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c406386-9fe0-4f3b-b65c-39e706bd53c7 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:39:19.306239 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c406386-9fe0-4f3b-b65c-39e706bd53c7 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:39:19.309728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb409c-cf19-43f7-8e4e-367280e84b19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:39:19.310355 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb409c-cf19-43f7-8e4e-367280e84b19 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:39:19.311318 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb409c-cf19-43f7-8e4e-367280e84b19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:39:19.311595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb409c-cf19-43f7-8e4e-367280e84b19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:39:19.320337 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb409c-cf19-43f7-8e4e-367280e84b19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:39:19.321756 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb409c-cf19-43f7-8e4e-367280e84b19 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:39:19.331198 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 757/1484] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:39:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:39:21.417596 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650169d-c2c3-409f-9254-f6e67a22943d None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:39:21.418183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650169d-c2c3-409f-9254-f6e67a22943d None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:39:21.418183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650169d-c2c3-409f-9254-f6e67a22943d None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:39:21.419532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 728/1485] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:39:21.425341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e24561-1cdb-421e-a169-ae835a2c410a None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:39:21.425771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e24561-1cdb-421e-a169-ae835a2c410a None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:39:21.425865 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e24561-1cdb-421e-a169-ae835a2c410a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:39:21.444841 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-f3e24561-1cdb-421e-a169-ae835a2c410a None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:39:21.446205 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-f3e24561-1cdb-421e-a169-ae835a2c410a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:39:21.454505 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 758/1486] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:39:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:39:21.493828 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650169d-c2c3-409f-9254-f6e67a22943d None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:39:21.504691 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650169d-c2c3-409f-9254-f6e67a22943d None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:39:21.508297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0394b5-fe27-48d0-bae2-bbf67c051d63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:39:21.508929 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0394b5-fe27-48d0-bae2-bbf67c051d63 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:39:21.509211 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0394b5-fe27-48d0-bae2-bbf67c051d63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:39:21.509487 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0394b5-fe27-48d0-bae2-bbf67c051d63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:39:21.518956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0394b5-fe27-48d0-bae2-bbf67c051d63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:39:21.520368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0394b5-fe27-48d0-bae2-bbf67c051d63 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:39:21.530934 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 729/1487] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:39:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:39:23.673091 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b99600-af91-4ed0-b307-6fc71f4ea7e3 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:39:23.673693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b99600-af91-4ed0-b307-6fc71f4ea7e3 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:39:23.673800 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b99600-af91-4ed0-b307-6fc71f4ea7e3 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:39:23.675153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 759/1488] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:39:23.680168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a33fdf-bff4-4daf-aa8d-1b3a35dfe1e8 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:39:23.680581 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a33fdf-bff4-4daf-aa8d-1b3a35dfe1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:39:23.680757 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a33fdf-bff4-4daf-aa8d-1b3a35dfe1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:39:23.698711 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-f0a33fdf-bff4-4daf-aa8d-1b3a35dfe1e8 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:39:23.700180 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f0a33fdf-bff4-4daf-aa8d-1b3a35dfe1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:39:23.708670 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 730/1489] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:39:23.750359 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b99600-af91-4ed0-b307-6fc71f4ea7e3 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:39:23.760915 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b99600-af91-4ed0-b307-6fc71f4ea7e3 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:39:23.764533 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6d040-899f-49fa-8337-c2a9de3943a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:39:23.765147 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6d040-899f-49fa-8337-c2a9de3943a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:39:23.765426 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6d040-899f-49fa-8337-c2a9de3943a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:39:23.765696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6d040-899f-49fa-8337-c2a9de3943a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:39:23.774246 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b6d040-899f-49fa-8337-c2a9de3943a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:39:23.775690 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b6d040-899f-49fa-8337-c2a9de3943a3 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:39:23.784855 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 760/1490] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:39:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:39:26.203243 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9a00b-c329-4e4d-adaa-5cbceeb1f8a3 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:39:26.203814 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9a00b-c329-4e4d-adaa-5cbceeb1f8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:39:26.203814 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9a00b-c329-4e4d-adaa-5cbceeb1f8a3 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:39:26.205151 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 731/1491] 10.222.0.15 () {54 vars in 926 bytes} [Mon Jun 27 18:39:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:39:26.210336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46940467-b766-4d86-bfb3-0afe72cb3f6c None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:39:26.210777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46940467-b766-4d86-bfb3-0afe72cb3f6c None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:39:26.210942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46940467-b766-4d86-bfb3-0afe72cb3f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:39:26.229123 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-46940467-b766-4d86-bfb3-0afe72cb3f6c None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:39:26.230307 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-46940467-b766-4d86-bfb3-0afe72cb3f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:39:26.238518 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 761/1492] 10.222.0.15 () {58 vars in 1023 bytes} [Mon Jun 27 18:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:39:26.277833 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9a00b-c329-4e4d-adaa-5cbceeb1f8a3 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:39:26.288990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9a00b-c329-4e4d-adaa-5cbceeb1f8a3 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:39:26.292418 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ec282-3cd3-4aee-bb83-111a22e7a195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:39:26.293034 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ec282-3cd3-4aee-bb83-111a22e7a195 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:39:26.293312 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ec282-3cd3-4aee-bb83-111a22e7a195 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:39:26.293584 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ec282-3cd3-4aee-bb83-111a22e7a195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:39:26.302293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ec282-3cd3-4aee-bb83-111a22e7a195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:39:26.303765 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ec282-3cd3-4aee-bb83-111a22e7a195 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:39:26.313106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 732/1493] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:39:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:39:36.224091 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=33 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:39:36.308748 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=33 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:41:26.875283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6d36b-09c4-4fa5-8a56-d9b1ce742aa9 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:41:26.875878 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6d36b-09c4-4fa5-8a56-d9b1ce742aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:41:26.875878 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6d36b-09c4-4fa5-8a56-d9b1ce742aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:41:26.894236 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-91c6d36b-09c4-4fa5-8a56-d9b1ce742aa9 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:41:26.895463 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-91c6d36b-09c4-4fa5-8a56-d9b1ce742aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:41:26.903439 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 762/1494] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:41:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:41:26.911675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50982222-073e-4e0e-a8a6-a86f199d7ec2 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:41:26.912168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50982222-073e-4e0e-a8a6-a86f199d7ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:41:26.912278 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50982222-073e-4e0e-a8a6-a86f199d7ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:41:26.929701 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-50982222-073e-4e0e-a8a6-a86f199d7ec2 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:41:26.930921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-50982222-073e-4e0e-a8a6-a86f199d7ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:41:26.939477 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 733/1495] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:41:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:41:26.943829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6d36b-09c4-4fa5-8a56-d9b1ce742aa9 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:41:26.953946 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6d36b-09c4-4fa5-8a56-d9b1ce742aa9 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:41:26.958433 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b3c01-aeb6-4e9c-8c69-8336a569381f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:41:26.959071 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b3c01-aeb6-4e9c-8c69-8336a569381f admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:41:26.959351 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b3c01-aeb6-4e9c-8c69-8336a569381f admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:41:26.959625 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b3c01-aeb6-4e9c-8c69-8336a569381f admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:41:26.960341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b3c01-aeb6-4e9c-8c69-8336a569381f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:41:26.960644 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b3c01-aeb6-4e9c-8c69-8336a569381f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:41:26.961646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: WARNING py.warnings [None req-049b3c01-aeb6-4e9c-8c69-8336a569381f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 18:41:26.961646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: warnings.warn(msg) Jun 27 18:41:26.961646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: Jun 27 18:41:26.962551 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b3c01-aeb6-4e9c-8c69-8336a569381f admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:41:26.973821 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 763/1496] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:41:26.982327 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50982222-073e-4e0e-a8a6-a86f199d7ec2 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:41:26.991514 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50982222-073e-4e0e-a8a6-a86f199d7ec2 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:41:26.995997 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc5509-0cd3-4769-a31b-756362744b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:41:26.996631 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc5509-0cd3-4769-a31b-756362744b69 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:41:26.997881 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc5509-0cd3-4769-a31b-756362744b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:41:26.998208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc5509-0cd3-4769-a31b-756362744b69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:41:26.998882 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bc5509-0cd3-4769-a31b-756362744b69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:41:27.000676 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: WARNING py.warnings [None req-31bc5509-0cd3-4769-a31b-756362744b69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 18:41:27.000676 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: warnings.warn(msg) Jun 27 18:41:27.000676 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: Jun 27 18:41:27.001863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bc5509-0cd3-4769-a31b-756362744b69 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:41:27.118826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 734/1497] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:41:26 2022] POST /identity/v3/projects => generated 379 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:41:27.123723 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b3c01-aeb6-4e9c-8c69-8336a569381f admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:41:27.133613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b3c01-aeb6-4e9c-8c69-8336a569381f admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:41:27.189229 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2880171-7b13-4409-a81f-2a2872f89e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:41:27.189879 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2880171-7b13-4409-a81f-2a2872f89e09 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:41:27.190159 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2880171-7b13-4409-a81f-2a2872f89e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:41:27.190454 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2880171-7b13-4409-a81f-2a2872f89e09 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:41:27.191141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2880171-7b13-4409-a81f-2a2872f89e09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:41:27.192148 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: WARNING py.warnings [None req-c2880171-7b13-4409-a81f-2a2872f89e09 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 18:41:27.192148 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: warnings.warn(msg) Jun 27 18:41:27.192148 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: Jun 27 18:41:27.193333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2880171-7b13-4409-a81f-2a2872f89e09 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:41:27.265338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 764/1498] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:41:27 2022] POST /identity/v3/users => generated 290 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:41:27.269608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc5509-0cd3-4769-a31b-756362744b69 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:41:27.279986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc5509-0cd3-4769-a31b-756362744b69 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:41:27.284593 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c74727-bd39-4540-856d-f522de71d5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:41:27.285245 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c74727-bd39-4540-856d-f522de71d5c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:41:27.285527 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c74727-bd39-4540-856d-f522de71d5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:41:27.285799 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c74727-bd39-4540-856d-f522de71d5c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:41:27.286587 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c74727-bd39-4540-856d-f522de71d5c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:41:27.287650 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: WARNING py.warnings [None req-d6c74727-bd39-4540-856d-f522de71d5c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 18:41:27.287650 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: warnings.warn(msg) Jun 27 18:41:27.287650 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: Jun 27 18:41:27.288531 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c74727-bd39-4540-856d-f522de71d5c3 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:41:27.301891 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 735/1499] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:41:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:41:27.305877 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2880171-7b13-4409-a81f-2a2872f89e09 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:41:27.316439 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2880171-7b13-4409-a81f-2a2872f89e09 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:41:27.320954 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d3858-b4bf-4c33-ad77-0ebe7f3f553a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:41:27.321595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d3858-b4bf-4c33-ad77-0ebe7f3f553a admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:41:27.321877 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d3858-b4bf-4c33-ad77-0ebe7f3f553a admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:41:27.322150 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d3858-b4bf-4c33-ad77-0ebe7f3f553a admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:41:27.322911 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d3858-b4bf-4c33-ad77-0ebe7f3f553a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:41:27.323911 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: WARNING py.warnings [None req-135d3858-b4bf-4c33-ad77-0ebe7f3f553a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 18:41:27.323911 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: warnings.warn(msg) Jun 27 18:41:27.323911 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: Jun 27 18:41:27.324959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d3858-b4bf-4c33-ad77-0ebe7f3f553a admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:41:27.337856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 765/1500] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:41:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:41:27.341908 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c74727-bd39-4540-856d-f522de71d5c3 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:41:27.351658 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c74727-bd39-4540-856d-f522de71d5c3 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:41:27.356765 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c9467-8f2c-4ca9-9c73-7fcdf45a0c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:41:27.357705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c9467-8f2c-4ca9-9c73-7fcdf45a0c4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:41:27.357985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c9467-8f2c-4ca9-9c73-7fcdf45a0c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:41:27.358259 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c9467-8f2c-4ca9-9c73-7fcdf45a0c4c admin admin] PATH_INFO: `/v3/projects/3fa4fd96ae0f47d29eca83437621e8ef/users/bd89080c42044b228eff805fe8b5c706/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:41:27.373585 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05c9467-8f2c-4ca9-9c73-7fcdf45a0c4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fa4fd96ae0f47d29eca83437621e8ef, user_id=bd89080c42044b228eff805fe8b5c706, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:41:27.374624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: WARNING py.warnings [None req-a05c9467-8f2c-4ca9-9c73-7fcdf45a0c4c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 18:41:27.374624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: warnings.warn(msg) Jun 27 18:41:27.374624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: Jun 27 18:41:27.376556 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05c9467-8f2c-4ca9-9c73-7fcdf45a0c4c admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:41:27.438667 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 736/1501] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:41:27 2022] PUT /identity/v3/projects/3fa4fd96ae0f47d29eca83437621e8ef/users/bd89080c42044b228eff805fe8b5c706/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:41:27.449976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3bd56-e5cd-4a40-8c22-7dfec1dde4d8 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:41:27.450280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3bd56-e5cd-4a40-8c22-7dfec1dde4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:41:27.450629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3bd56-e5cd-4a40-8c22-7dfec1dde4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:41:27.477906 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-9bb3bd56-e5cd-4a40-8c22-7dfec1dde4d8 None None] MFA Rules not processed for user `bd89080c42044b228eff805fe8b5c706`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:41:27.479136 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-9bb3bd56-e5cd-4a40-8c22-7dfec1dde4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:41:27.543213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 766/1502] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:41:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:41:27.556850 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea00c5-6637-414f-ae57-5eb059517ee8 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:41:27.557135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea00c5-6637-414f-ae57-5eb059517ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:41:27.557406 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea00c5-6637-414f-ae57-5eb059517ee8 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:41:27.558787 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 737/1503] 10.222.0.15 () {54 vars in 963 bytes} [Mon Jun 27 18:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:41:27.563530 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbe679-8ecf-49a9-830e-ea873b353830 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:41:27.563817 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbe679-8ecf-49a9-830e-ea873b353830 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:41:27.564101 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbe679-8ecf-49a9-830e-ea873b353830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:41:27.581378 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-b6dbe679-8ecf-49a9-830e-ea873b353830 None None] MFA Rules not processed for user `c4f6f8e0a0cb4c9a9dd0d99fc0022235`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:41:27.582611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-b6dbe679-8ecf-49a9-830e-ea873b353830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:41:27.626066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:41:27.633185 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 767/1504] 10.222.0.15 () {58 vars in 1060 bytes} [Mon Jun 27 18:41:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:41:27.644331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea00c5-6637-414f-ae57-5eb059517ee8 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:41:27.651532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea00c5-6637-414f-ae57-5eb059517ee8 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:41:27.655781 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ad202-c9d7-4ea4-b02d-af94a40fa9c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:41:27.656401 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ad202-c9d7-4ea4-b02d-af94a40fa9c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:41:27.656682 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ad202-c9d7-4ea4-b02d-af94a40fa9c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:41:27.656955 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ad202-c9d7-4ea4-b02d-af94a40fa9c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:41:27.665437 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ad202-c9d7-4ea4-b02d-af94a40fa9c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:41:27.666865 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ad202-c9d7-4ea4-b02d-af94a40fa9c9 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:41:27.676191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 738/1505] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:41:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:41:27.718770 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad79809-0df6-4578-9a3e-8ff28a1c961f None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:41:27.719064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad79809-0df6-4578-9a3e-8ff28a1c961f None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:41:27.719348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad79809-0df6-4578-9a3e-8ff28a1c961f None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:41:27.720636 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 768/1506] 10.222.0.15 () {52 vars in 829 bytes} [Mon Jun 27 18:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:41:27.724393 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ad202-c9d7-4ea4-b02d-af94a40fa9c9 service cinder] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:41:27.787314 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ad202-c9d7-4ea4-b02d-af94a40fa9c9 service cinder] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:41:27.853811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4570f-76d7-456b-9f1a-3ad17025cd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:41:27.853811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4570f-76d7-456b-9f1a-3ad17025cd21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:41:27.853811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4570f-76d7-456b-9f1a-3ad17025cd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:41:27.853811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4570f-76d7-456b-9f1a-3ad17025cd21 admin admin] PATH_INFO: `/v3/users/bd89080c42044b228eff805fe8b5c706` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:41:27.854263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4570f-76d7-456b-9f1a-3ad17025cd21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd89080c42044b228eff805fe8b5c706)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:41:27.854263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: WARNING py.warnings [None req-2ab4570f-76d7-456b-9f1a-3ad17025cd21 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 18:41:27.854263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: warnings.warn(msg) Jun 27 18:41:27.854263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: Jun 27 18:41:27.854263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4570f-76d7-456b-9f1a-3ad17025cd21 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:41:27.976513 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2ab4570f-76d7-456b-9f1a-3ad17025cd21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd89080c42044b228eff805fe8b5c706'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:41:27.989143 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2ab4570f-76d7-456b-9f1a-3ad17025cd21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd89080c42044b228eff805fe8b5c706'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:41:27.989916 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2ab4570f-76d7-456b-9f1a-3ad17025cd21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd89080c42044b228eff805fe8b5c706'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:41:28.009437 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2ab4570f-76d7-456b-9f1a-3ad17025cd21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd89080c42044b228eff805fe8b5c706'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:41:28.014067 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:41:28.112699 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2ab4570f-76d7-456b-9f1a-3ad17025cd21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd89080c42044b228eff805fe8b5c706'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:41:28.119101 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 739/1507] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:41:27 2022] DELETE /identity/v3/users/bd89080c42044b228eff805fe8b5c706 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:41:28.123854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9323ac-1788-474d-bb3f-c85ec5b73f82 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:41:28.125965 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9323ac-1788-474d-bb3f-c85ec5b73f82 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:41:28.125965 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9323ac-1788-474d-bb3f-c85ec5b73f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:41:28.141812 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-4a9323ac-1788-474d-bb3f-c85ec5b73f82 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:41:28.143054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-4a9323ac-1788-474d-bb3f-c85ec5b73f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:41:28.201699 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 769/1508] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:41:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:41:28.211122 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4570f-76d7-456b-9f1a-3ad17025cd21 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:41:28.212551 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-2ab4570f-76d7-456b-9f1a-3ad17025cd21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:41:28.268902 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4570f-76d7-456b-9f1a-3ad17025cd21 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:41:28.272516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a36f6-b7c2-4967-8ec8-6b85e3e76f44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:41:28.273159 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a36f6-b7c2-4967-8ec8-6b85e3e76f44 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:41:28.273442 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a36f6-b7c2-4967-8ec8-6b85e3e76f44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:41:28.273717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a36f6-b7c2-4967-8ec8-6b85e3e76f44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:41:28.283317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575a36f6-b7c2-4967-8ec8-6b85e3e76f44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:41:28.284742 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575a36f6-b7c2-4967-8ec8-6b85e3e76f44 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:41:28.298842 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 740/1509] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:41:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:41:28.799941 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9323ac-1788-474d-bb3f-c85ec5b73f82 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:41:28.804702 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-4a9323ac-1788-474d-bb3f-c85ec5b73f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:41:28.813774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9323ac-1788-474d-bb3f-c85ec5b73f82 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:41:28.818588 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17523fff-b646-4e7c-9915-b53981bae7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:41:28.819386 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17523fff-b646-4e7c-9915-b53981bae7e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:41:28.819667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17523fff-b646-4e7c-9915-b53981bae7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:41:28.819939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17523fff-b646-4e7c-9915-b53981bae7e0 admin admin] PATH_INFO: `/v3/projects/3fa4fd96ae0f47d29eca83437621e8ef` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:41:28.821092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17523fff-b646-4e7c-9915-b53981bae7e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fa4fd96ae0f47d29eca83437621e8ef)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:41:28.824145 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: WARNING py.warnings [None req-17523fff-b646-4e7c-9915-b53981bae7e0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 18:41:28.824145 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: warnings.warn(msg) Jun 27 18:41:28.824145 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: Jun 27 18:41:28.824145 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17523fff-b646-4e7c-9915-b53981bae7e0 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:41:28.948286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-17523fff-b646-4e7c-9915-b53981bae7e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fa4fd96ae0f47d29eca83437621e8ef', 'request_id': 'req-17523fff-b646-4e7c-9915-b53981bae7e0'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:41:29.009018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-17523fff-b646-4e7c-9915-b53981bae7e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fa4fd96ae0f47d29eca83437621e8ef', 'request_id': 'req-17523fff-b646-4e7c-9915-b53981bae7e0'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:41:29.023167 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-17523fff-b646-4e7c-9915-b53981bae7e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fa4fd96ae0f47d29eca83437621e8ef', 'request_id': 'req-17523fff-b646-4e7c-9915-b53981bae7e0'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:41:29.034097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-17523fff-b646-4e7c-9915-b53981bae7e0 admin admin] The token cache is being invalidate because project 3fa4fd96ae0f47d29eca83437621e8ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:41:29.034407 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-17523fff-b646-4e7c-9915-b53981bae7e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:41:29.035127 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-17523fff-b646-4e7c-9915-b53981bae7e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:41:29.036685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 770/1510] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:41:28 2022] DELETE /identity/v3/projects/3fa4fd96ae0f47d29eca83437621e8ef => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:41:38.294198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:41:39.031123 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:43:06.319758 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55a6b9-9a8b-40ea-bd44-f8e307f82dc2 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:06.320317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55a6b9-9a8b-40ea-bd44-f8e307f82dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:06.320317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55a6b9-9a8b-40ea-bd44-f8e307f82dc2 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:06.321594 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 741/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 18:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:43:06.326420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebb1ac-7940-42d3-beca-865b71746100 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:06.326865 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebb1ac-7940-42d3-beca-865b71746100 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:06.327073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ebb1ac-7940-42d3-beca-865b71746100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:06.345176 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-a4ebb1ac-7940-42d3-beca-865b71746100 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:43:06.346366 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-a4ebb1ac-7940-42d3-beca-865b71746100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:43:06.410212 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 771/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 18:43:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:43:06.565857 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55a6b9-9a8b-40ea-bd44-f8e307f82dc2 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:43:06.567637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-8b55a6b9-9a8b-40ea-bd44-f8e307f82dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:43:06.642963 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55a6b9-9a8b-40ea-bd44-f8e307f82dc2 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:43:06.648260 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f715b45-15d2-4ae2-9955-a3bccb1de35e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:43:06.648894 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f715b45-15d2-4ae2-9955-a3bccb1de35e service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:06.649176 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f715b45-15d2-4ae2-9955-a3bccb1de35e service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:06.649452 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f715b45-15d2-4ae2-9955-a3bccb1de35e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:06.657634 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f715b45-15d2-4ae2-9955-a3bccb1de35e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:43:06.659262 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f715b45-15d2-4ae2-9955-a3bccb1de35e service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:43:06.669057 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 742/1513] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:43:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:43:07.499238 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dfa9f-14bf-4da0-8d3d-5b7e5300fd18 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:07.499818 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dfa9f-14bf-4da0-8d3d-5b7e5300fd18 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:07.499929 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dfa9f-14bf-4da0-8d3d-5b7e5300fd18 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:07.501532 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 772/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 18:43:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:43:07.506464 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff408be-53e8-4611-a5df-f8d80f5babfd None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:07.506871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff408be-53e8-4611-a5df-f8d80f5babfd None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:07.507019 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff408be-53e8-4611-a5df-f8d80f5babfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:07.526105 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-2ff408be-53e8-4611-a5df-f8d80f5babfd None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:43:07.527328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-2ff408be-53e8-4611-a5df-f8d80f5babfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:43:07.535247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 743/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 18:43:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:43:07.689055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751dfa9f-14bf-4da0-8d3d-5b7e5300fd18 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:43:07.703480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751dfa9f-14bf-4da0-8d3d-5b7e5300fd18 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:43:07.708822 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96158f-410a-4fab-88fd-fa9a718adde8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:43:07.709449 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96158f-410a-4fab-88fd-fa9a718adde8 service nova] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:07.709738 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96158f-410a-4fab-88fd-fa9a718adde8 service nova] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:07.710013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96158f-410a-4fab-88fd-fa9a718adde8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:07.718586 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b96158f-410a-4fab-88fd-fa9a718adde8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:43:07.720185 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b96158f-410a-4fab-88fd-fa9a718adde8 service nova] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:43:07.730214 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 773/1516] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:43:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:43:08.832947 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6c2b1-1fc5-4815-a43a-606d18b89029 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:08.833522 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6c2b1-1fc5-4815-a43a-606d18b89029 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:08.833522 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6c2b1-1fc5-4815-a43a-606d18b89029 None None] PATH_INFO: `/v3` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:08.834869 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 744/1517] 10.222.0.33 () {54 vars in 925 bytes} [Mon Jun 27 18:43:08 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:43:08.860538 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d5218-5b16-4e55-ac5b-657b63959544 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:08.861002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d5218-5b16-4e55-ac5b-657b63959544 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:08.861106 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d5218-5b16-4e55-ac5b-657b63959544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:08.879663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-726d5218-5b16-4e55-ac5b-657b63959544 None None] MFA Rules not processed for user `3cd6f0a1286243e4add692db01028205`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:43:08.880878 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-726d5218-5b16-4e55-ac5b-657b63959544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:43:08.942006 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 774/1518] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Jun 27 18:43:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:43:12.850561 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6814030-7a98-4ef7-801a-596e1f842376 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:12.851094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6814030-7a98-4ef7-801a-596e1f842376 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:12.851094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6814030-7a98-4ef7-801a-596e1f842376 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:12.852461 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 745/1519] 10.222.0.15 () {54 vars in 960 bytes} [Mon Jun 27 18:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:43:12.857645 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b3a12-7105-49b0-a367-703980ee70bc None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:12.858083 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b3a12-7105-49b0-a367-703980ee70bc None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:12.858211 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b3a12-7105-49b0-a367-703980ee70bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:12.877028 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-1d1b3a12-7105-49b0-a367-703980ee70bc None None] MFA Rules not processed for user `3cd6f0a1286243e4add692db01028205`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:43:12.878230 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-1d1b3a12-7105-49b0-a367-703980ee70bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:43:12.887333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 775/1520] 10.222.0.15 () {58 vars in 1057 bytes} [Mon Jun 27 18:43:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:43:12.894743 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6814030-7a98-4ef7-801a-596e1f842376 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:43:12.908373 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6814030-7a98-4ef7-801a-596e1f842376 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:43:12.913190 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762df1f-7c94-46f9-9483-4ee53ef71b08 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cd6f0a1286243e4add692db01028205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:43:12.913805 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762df1f-7c94-46f9-9483-4ee53ef71b08 service placement] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:12.914084 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762df1f-7c94-46f9-9483-4ee53ef71b08 service placement] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:12.914354 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762df1f-7c94-46f9-9483-4ee53ef71b08 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:12.922902 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762df1f-7c94-46f9-9483-4ee53ef71b08 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:43:12.924467 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762df1f-7c94-46f9-9483-4ee53ef71b08 service placement] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:43:12.934671 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 746/1521] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:43:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:43:18.951772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1647c8-7456-455d-acd9-b9d7603d997d None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:18.952352 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1647c8-7456-455d-acd9-b9d7603d997d None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:18.952352 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1647c8-7456-455d-acd9-b9d7603d997d None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:18.953696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 776/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 18:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:43:18.958684 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d22b1-52a8-4f64-8995-a63b5e84b065 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:18.959135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d22b1-52a8-4f64-8995-a63b5e84b065 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:18.959240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d22b1-52a8-4f64-8995-a63b5e84b065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:18.977845 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-471d22b1-52a8-4f64-8995-a63b5e84b065 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:43:18.981627 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-471d22b1-52a8-4f64-8995-a63b5e84b065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:43:18.987853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 747/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 18:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:43:19.141382 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1647c8-7456-455d-acd9-b9d7603d997d None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:43:19.156633 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1647c8-7456-455d-acd9-b9d7603d997d None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:43:19.162067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639c51b-c5f1-48c4-ba7a-d34efb7b5d7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:43:19.162733 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639c51b-c5f1-48c4-ba7a-d34efb7b5d7a service nova] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:19.163016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639c51b-c5f1-48c4-ba7a-d34efb7b5d7a service nova] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:19.163294 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639c51b-c5f1-48c4-ba7a-d34efb7b5d7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:19.171703 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639c51b-c5f1-48c4-ba7a-d34efb7b5d7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:43:19.173289 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639c51b-c5f1-48c4-ba7a-d34efb7b5d7a service nova] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:43:19.182732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 777/1524] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:43:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:43:20.004850 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb81dae-85ec-459f-bd82-ad8fa6d47d25 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:20.005393 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb81dae-85ec-459f-bd82-ad8fa6d47d25 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:20.005393 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb81dae-85ec-459f-bd82-ad8fa6d47d25 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:20.006737 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 748/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 18:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:43:20.011502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b826c9-bb07-42c6-95de-8acb43337aae None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:20.012248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b826c9-bb07-42c6-95de-8acb43337aae None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:20.012248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b826c9-bb07-42c6-95de-8acb43337aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:20.029593 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-99b826c9-bb07-42c6-95de-8acb43337aae None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:43:20.030825 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-99b826c9-bb07-42c6-95de-8acb43337aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:43:20.038599 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 778/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 18:43:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:43:20.189569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb81dae-85ec-459f-bd82-ad8fa6d47d25 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:43:20.204462 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb81dae-85ec-459f-bd82-ad8fa6d47d25 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:43:20.209776 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081eb054-ce12-4cff-9921-dddf33d87eed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:43:20.210416 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081eb054-ce12-4cff-9921-dddf33d87eed service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:20.210702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081eb054-ce12-4cff-9921-dddf33d87eed service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:20.210972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081eb054-ce12-4cff-9921-dddf33d87eed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:20.219996 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081eb054-ce12-4cff-9921-dddf33d87eed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:43:20.222726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081eb054-ce12-4cff-9921-dddf33d87eed service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:43:20.231294 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 749/1527] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:43:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:43:24.092582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172878e-80f6-49f3-bea2-bb5fcadcb097 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:24.093184 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172878e-80f6-49f3-bea2-bb5fcadcb097 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:24.093184 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172878e-80f6-49f3-bea2-bb5fcadcb097 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:24.094559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 779/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 18:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:43:24.099849 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762197f-ffbe-432f-9b7f-a8a2464bdfbc None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:24.100277 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762197f-ffbe-432f-9b7f-a8a2464bdfbc None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:24.100439 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762197f-ffbe-432f-9b7f-a8a2464bdfbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:24.119310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-b762197f-ffbe-432f-9b7f-a8a2464bdfbc None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:43:24.120554 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-b762197f-ffbe-432f-9b7f-a8a2464bdfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:43:24.128647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 750/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 18:43:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:43:24.165368 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172878e-80f6-49f3-bea2-bb5fcadcb097 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:43:24.167058 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-b172878e-80f6-49f3-bea2-bb5fcadcb097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:43:24.222903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172878e-80f6-49f3-bea2-bb5fcadcb097 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:43:24.226587 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0201a6-2068-4c98-8ef5-9b1812a9661f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:43:24.227230 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0201a6-2068-4c98-8ef5-9b1812a9661f service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:24.227516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0201a6-2068-4c98-8ef5-9b1812a9661f service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:24.227819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0201a6-2068-4c98-8ef5-9b1812a9661f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:24.237359 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0201a6-2068-4c98-8ef5-9b1812a9661f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:43:24.238829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0201a6-2068-4c98-8ef5-9b1812a9661f service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:43:24.243637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:43:24.249115 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 780/1530] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:43:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:43:26.271314 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bdd01-2a49-4cbc-a0dc-e2a4819e06ec None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:26.271858 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bdd01-2a49-4cbc-a0dc-e2a4819e06ec None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:26.271973 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bdd01-2a49-4cbc-a0dc-e2a4819e06ec None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:26.273293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 751/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 18:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:43:26.278816 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fe4f0-acb4-43db-a944-57b82608e1ca None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:26.279224 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fe4f0-acb4-43db-a944-57b82608e1ca None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:26.279369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fe4f0-acb4-43db-a944-57b82608e1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:26.297911 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-c47fe4f0-acb4-43db-a944-57b82608e1ca None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:43:26.299140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c47fe4f0-acb4-43db-a944-57b82608e1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:43:26.306890 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 781/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 18:43:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:43:26.343396 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bdd01-2a49-4cbc-a0dc-e2a4819e06ec None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:43:26.353890 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bdd01-2a49-4cbc-a0dc-e2a4819e06ec None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:43:26.358496 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc01f95-66dd-4dea-97dc-b921c22d6a3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:43:26.359124 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc01f95-66dd-4dea-97dc-b921c22d6a3f service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:26.359401 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc01f95-66dd-4dea-97dc-b921c22d6a3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:26.359680 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc01f95-66dd-4dea-97dc-b921c22d6a3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:26.369134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc01f95-66dd-4dea-97dc-b921c22d6a3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:43:26.371111 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc01f95-66dd-4dea-97dc-b921c22d6a3f service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:43:26.380743 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 752/1533] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:43:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:43:29.876336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf15d1-e6be-4b56-895d-594476c631a1 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:29.876918 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf15d1-e6be-4b56-895d-594476c631a1 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:29.876918 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf15d1-e6be-4b56-895d-594476c631a1 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:29.878303 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 782/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 18:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:43:29.884433 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64c992-a4af-48a1-8996-9a22135c1890 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:29.884845 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64c992-a4af-48a1-8996-9a22135c1890 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:29.884995 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64c992-a4af-48a1-8996-9a22135c1890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:29.891516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=62 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:43:29.903390 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-9f64c992-a4af-48a1-8996-9a22135c1890 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:43:29.906488 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-9f64c992-a4af-48a1-8996-9a22135c1890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:43:29.912653 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 753/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 18:43:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:43:30.222775 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf15d1-e6be-4b56-895d-594476c631a1 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:43:30.224430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-6daf15d1-e6be-4b56-895d-594476c631a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:43:30.281062 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf15d1-e6be-4b56-895d-594476c631a1 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:43:30.284734 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d456c8-a7d1-491b-8a8e-357a3b8b381f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:43:30.285368 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d456c8-a7d1-491b-8a8e-357a3b8b381f service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:30.285654 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d456c8-a7d1-491b-8a8e-357a3b8b381f service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:30.285934 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d456c8-a7d1-491b-8a8e-357a3b8b381f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:30.295546 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d456c8-a7d1-491b-8a8e-357a3b8b381f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:43:30.296979 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d456c8-a7d1-491b-8a8e-357a3b8b381f service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:43:30.306342 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 783/1536] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:43:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:43:32.375756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca73da9-7fa0-4f83-8a6e-956080a3746c None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:32.376234 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca73da9-7fa0-4f83-8a6e-956080a3746c None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:32.376345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca73da9-7fa0-4f83-8a6e-956080a3746c None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:32.377694 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 754/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 18:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:43:32.382938 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370c731-953c-457f-bec0-d4a1638d42ae None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:32.383351 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370c731-953c-457f-bec0-d4a1638d42ae None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:32.383504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370c731-953c-457f-bec0-d4a1638d42ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:32.402300 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-5370c731-953c-457f-bec0-d4a1638d42ae None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:43:32.403522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-5370c731-953c-457f-bec0-d4a1638d42ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:43:32.411254 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 784/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 18:43:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:43:32.711776 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca73da9-7fa0-4f83-8a6e-956080a3746c None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:43:32.713460 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-3ca73da9-7fa0-4f83-8a6e-956080a3746c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:43:32.726495 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca73da9-7fa0-4f83-8a6e-956080a3746c None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:43:32.730700 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7331db-52bb-4adf-81db-605f2a086144 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:43:32.731354 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7331db-52bb-4adf-81db-605f2a086144 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:43:32.731639 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7331db-52bb-4adf-81db-605f2a086144 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:43:32.731913 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7331db-52bb-4adf-81db-605f2a086144 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:43:32.741193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7331db-52bb-4adf-81db-605f2a086144 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:43:32.742673 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7331db-52bb-4adf-81db-605f2a086144 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:43:32.753033 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 755/1539] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:43:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:43:42.396420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=32 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:43:42.748905 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=8 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:00.044428 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a56e35-147e-41bf-9a72-d42098364172 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.045050 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a56e35-147e-41bf-9a72-d42098364172 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.045050 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a56e35-147e-41bf-9a72-d42098364172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.070713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-80a56e35-147e-41bf-9a72-d42098364172 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:00.072903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-80a56e35-147e-41bf-9a72-d42098364172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:00.085063 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 785/1540] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:45:00.097544 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218332e-e17f-4687-9a90-aa1b25462404 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.097544 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218332e-e17f-4687-9a90-aa1b25462404 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.097544 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218332e-e17f-4687-9a90-aa1b25462404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.123442 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-6218332e-e17f-4687-9a90-aa1b25462404 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:00.124677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-6218332e-e17f-4687-9a90-aa1b25462404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:00.134944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 756/1541] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:00.139638 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a56e35-147e-41bf-9a72-d42098364172 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:00.155155 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a56e35-147e-41bf-9a72-d42098364172 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:00.160618 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323712ab-295d-43f8-af51-e58dc67d7805 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.160999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323712ab-295d-43f8-af51-e58dc67d7805 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.161192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323712ab-295d-43f8-af51-e58dc67d7805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.161583 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c174f-76d7-4368-9741-079620f4f1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:00.162641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c174f-76d7-4368-9741-079620f4f1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.163177 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c174f-76d7-4368-9741-079620f4f1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.163765 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c174f-76d7-4368-9741-079620f4f1f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.164750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c174f-76d7-4368-9741-079620f4f1f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:45:00.165290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c174f-76d7-4368-9741-079620f4f1f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:00.167602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c174f-76d7-4368-9741-079620f4f1f1 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:00.186709 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 786/1542] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:45:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:45:00.195419 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c174f-76d7-4368-9741-079620f4f1f1 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:00.195660 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-323712ab-295d-43f8-af51-e58dc67d7805 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:00.196725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-323712ab-295d-43f8-af51-e58dc67d7805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:00.217505 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c174f-76d7-4368-9741-079620f4f1f1 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:00.221524 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 757/1543] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:00.225375 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c565193-89b5-4892-9b1f-06fa2963bed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:00.226445 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c565193-89b5-4892-9b1f-06fa2963bed9 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.226949 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c565193-89b5-4892-9b1f-06fa2963bed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.227430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c565193-89b5-4892-9b1f-06fa2963bed9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.228457 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c565193-89b5-4892-9b1f-06fa2963bed9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:00.229727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: WARNING py.warnings [None req-4c565193-89b5-4892-9b1f-06fa2963bed9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 18:45:00.229727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: warnings.warn(msg) Jun 27 18:45:00.229727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: Jun 27 18:45:00.231377 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c565193-89b5-4892-9b1f-06fa2963bed9 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:00.237380 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750146f-412d-46f9-9486-4263b4e0ae96 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.237380 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750146f-412d-46f9-9486-4263b4e0ae96 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.237380 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750146f-412d-46f9-9486-4263b4e0ae96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.271248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-a750146f-412d-46f9-9486-4263b4e0ae96 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:00.272483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-a750146f-412d-46f9-9486-4263b4e0ae96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:00.288884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 758/1544] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:45:00.294189 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750146f-412d-46f9-9486-4263b4e0ae96 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:00.305935 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750146f-412d-46f9-9486-4263b4e0ae96 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:00.315536 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e383ad7-d7c8-4d0a-8a39-21d5abed2cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:00.316218 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e383ad7-d7c8-4d0a-8a39-21d5abed2cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.316504 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e383ad7-d7c8-4d0a-8a39-21d5abed2cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.316779 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e383ad7-d7c8-4d0a-8a39-21d5abed2cc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.317526 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e383ad7-d7c8-4d0a-8a39-21d5abed2cc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:45:00.317835 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e383ad7-d7c8-4d0a-8a39-21d5abed2cc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:00.318885 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: WARNING py.warnings [None req-6e383ad7-d7c8-4d0a-8a39-21d5abed2cc8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 18:45:00.318885 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: warnings.warn(msg) Jun 27 18:45:00.318885 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: Jun 27 18:45:00.319790 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e383ad7-d7c8-4d0a-8a39-21d5abed2cc8 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:00.333317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 759/1545] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:45:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:45:00.334762 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e4814-d165-435a-8fce-2c47ee4877b8 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.335057 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e4814-d165-435a-8fce-2c47ee4877b8 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.335337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e4814-d165-435a-8fce-2c47ee4877b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.347188 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 787/1546] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:00 2022] POST /identity/v3/projects => generated 371 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:45:00.349050 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4b265-5bc3-4c00-8d5b-68bf70c0207b None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.349343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4b265-5bc3-4c00-8d5b-68bf70c0207b None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.349623 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4b265-5bc3-4c00-8d5b-68bf70c0207b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.375648 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-004e4814-d165-435a-8fce-2c47ee4877b8 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:00.376908 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-004e4814-d165-435a-8fce-2c47ee4877b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:00.385194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-d8b4b265-5bc3-4c00-8d5b-68bf70c0207b None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:00.391854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-d8b4b265-5bc3-4c00-8d5b-68bf70c0207b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:00.490509 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 760/1547] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:00.492303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e4814-d165-435a-8fce-2c47ee4877b8 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:00.504441 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e4814-d165-435a-8fce-2c47ee4877b8 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:00.505505 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 788/1548] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:00.506316 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4b265-5bc3-4c00-8d5b-68bf70c0207b None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:00.514922 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca77ed-2071-4d6e-a5c6-f986ebc0bf68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:00.516313 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca77ed-2071-4d6e-a5c6-f986ebc0bf68 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.516755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca77ed-2071-4d6e-a5c6-f986ebc0bf68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.517172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca77ed-2071-4d6e-a5c6-f986ebc0bf68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.518644 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca77ed-2071-4d6e-a5c6-f986ebc0bf68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:00.520733 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca77ed-2071-4d6e-a5c6-f986ebc0bf68 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:00.524027 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4b265-5bc3-4c00-8d5b-68bf70c0207b None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:00.530376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0349d-35c6-42f4-b0d0-76e317331994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:00.531204 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0349d-35c6-42f4-b0d0-76e317331994 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.531594 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0349d-35c6-42f4-b0d0-76e317331994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.531975 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0349d-35c6-42f4-b0d0-76e317331994 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.533372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc0349d-35c6-42f4-b0d0-76e317331994 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:00.535967 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc0349d-35c6-42f4-b0d0-76e317331994 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:00.629738 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 761/1549] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:00 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:00.631053 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc400a1-db1b-4e5b-9bf5-17fcd7c5e0a1 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.631346 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc400a1-db1b-4e5b-9bf5-17fcd7c5e0a1 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.631623 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc400a1-db1b-4e5b-9bf5-17fcd7c5e0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.649691 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-ecc400a1-db1b-4e5b-9bf5-17fcd7c5e0a1 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:00.651240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-ecc400a1-db1b-4e5b-9bf5-17fcd7c5e0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:00.656732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 789/1550] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:00 2022] POST /identity/v3/users => generated 286 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:00.656732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1fc13-002e-47f0-b8fe-002433a9154a None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.656732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1fc13-002e-47f0-b8fe-002433a9154a None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.656732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1fc13-002e-47f0-b8fe-002433a9154a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.671829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-5ab1fc13-002e-47f0-b8fe-002433a9154a None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:00.673031 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-5ab1fc13-002e-47f0-b8fe-002433a9154a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:00.712170 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 762/1551] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:00.713486 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b9492-3072-4eb0-adf6-390fe55958f1 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.713776 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b9492-3072-4eb0-adf6-390fe55958f1 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.714049 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b9492-3072-4eb0-adf6-390fe55958f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.732264 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-6a8b9492-3072-4eb0-adf6-390fe55958f1 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:00.733733 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-6a8b9492-3072-4eb0-adf6-390fe55958f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:00.739612 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 790/1552] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:00.741122 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9821df5-4f1c-48f9-9bad-d5228c4b2601 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.741361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9821df5-4f1c-48f9-9bad-d5228c4b2601 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.741645 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9821df5-4f1c-48f9-9bad-d5228c4b2601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.746632 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 763/1553] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:00.746632 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26d905-21b2-4a32-b961-1e7a64bad18a None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.746632 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26d905-21b2-4a32-b961-1e7a64bad18a None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.746632 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26d905-21b2-4a32-b961-1e7a64bad18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.763122 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-9e26d905-21b2-4a32-b961-1e7a64bad18a None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:00.764327 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-9e26d905-21b2-4a32-b961-1e7a64bad18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:00.764693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-a9821df5-4f1c-48f9-9bad-d5228c4b2601 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:00.765872 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-a9821df5-4f1c-48f9-9bad-d5228c4b2601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:00.772793 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 764/1554] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:00.777752 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 791/1555] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:00.777917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49072224-6231-450f-bf61-477edd155e10 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.777917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49072224-6231-450f-bf61-477edd155e10 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.778104 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9821df5-4f1c-48f9-9bad-d5228c4b2601 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:00.778202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49072224-6231-450f-bf61-477edd155e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.787707 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9821df5-4f1c-48f9-9bad-d5228c4b2601 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:00.793239 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5b001-3343-46c4-b7da-2c915069ad73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:00.793949 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5b001-3343-46c4-b7da-2c915069ad73 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.794237 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5b001-3343-46c4-b7da-2c915069ad73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.794553 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5b001-3343-46c4-b7da-2c915069ad73 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.795242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f5b001-3343-46c4-b7da-2c915069ad73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:00.795645 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-49072224-6231-450f-bf61-477edd155e10 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:00.796832 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-49072224-6231-450f-bf61-477edd155e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:00.797155 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f5b001-3343-46c4-b7da-2c915069ad73 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:00.804611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 765/1556] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:00.805538 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49072224-6231-450f-bf61-477edd155e10 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:00.815949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49072224-6231-450f-bf61-477edd155e10 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:00.820624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164dcf96-f402-49d9-b707-0c724fe5c0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:00.821249 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164dcf96-f402-49d9-b707-0c724fe5c0b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.821533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164dcf96-f402-49d9-b707-0c724fe5c0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.822354 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164dcf96-f402-49d9-b707-0c724fe5c0b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.823141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164dcf96-f402-49d9-b707-0c724fe5c0b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:00.824763 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164dcf96-f402-49d9-b707-0c724fe5c0b3 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:00.834634 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 766/1557] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:00.835942 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bdbe9b-55f7-45b9-ba4b-dc2c673490c4 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.836237 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bdbe9b-55f7-45b9-ba4b-dc2c673490c4 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.836518 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bdbe9b-55f7-45b9-ba4b-dc2c673490c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.854802 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-45bdbe9b-55f7-45b9-ba4b-dc2c673490c4 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:00.856013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-45bdbe9b-55f7-45b9-ba4b-dc2c673490c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:00.864144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 767/1558] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:00.864896 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bdbe9b-55f7-45b9-ba4b-dc2c673490c4 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:00.873911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bdbe9b-55f7-45b9-ba4b-dc2c673490c4 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:00.878321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df28037-aa26-449a-8f44-63649b56ced1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:00.878968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df28037-aa26-449a-8f44-63649b56ced1 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.879246 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df28037-aa26-449a-8f44-63649b56ced1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.879524 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df28037-aa26-449a-8f44-63649b56ced1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.880275 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df28037-aa26-449a-8f44-63649b56ced1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:45:00.880527 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df28037-aa26-449a-8f44-63649b56ced1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:00.882083 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df28037-aa26-449a-8f44-63649b56ced1 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:00.885255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:00.894388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 768/1559] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:45:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:45:00.895598 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746efe4f-0146-4d13-9ba6-754c3ec3b262 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.895908 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746efe4f-0146-4d13-9ba6-754c3ec3b262 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.896184 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746efe4f-0146-4d13-9ba6-754c3ec3b262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.911277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 792/1560] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:00 2022] POST /identity/v3/users => generated 288 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:00.912363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5b001-3343-46c4-b7da-2c915069ad73 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:00.915393 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-746efe4f-0146-4d13-9ba6-754c3ec3b262 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:00.916721 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-746efe4f-0146-4d13-9ba6-754c3ec3b262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:00.922858 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5b001-3343-46c4-b7da-2c915069ad73 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:00.926146 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 769/1561] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:00.927758 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0907942-0f14-4f06-8437-fe61e50541fa None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.928108 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0907942-0f14-4f06-8437-fe61e50541fa None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.928390 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0907942-0f14-4f06-8437-fe61e50541fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.930933 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05af46-5a04-4b37-8656-4563f4d967a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:00.937793 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05af46-5a04-4b37-8656-4563f4d967a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.937793 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05af46-5a04-4b37-8656-4563f4d967a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.937793 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05af46-5a04-4b37-8656-4563f4d967a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.937793 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed05af46-5a04-4b37-8656-4563f4d967a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:45:00.937793 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed05af46-5a04-4b37-8656-4563f4d967a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:00.937793 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed05af46-5a04-4b37-8656-4563f4d967a1 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:00.946407 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-a0907942-0f14-4f06-8437-fe61e50541fa None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:00.948288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-a0907942-0f14-4f06-8437-fe61e50541fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:00.948605 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 793/1562] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:45:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:45:00.949952 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07a1dd-5a2f-4bd3-82ce-180472c353da None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.950242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07a1dd-5a2f-4bd3-82ce-180472c353da None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.950611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b07a1dd-5a2f-4bd3-82ce-180472c353da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.957536 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 770/1563] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:00.957665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:00.958282 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0907942-0f14-4f06-8437-fe61e50541fa None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:00.969219 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0907942-0f14-4f06-8437-fe61e50541fa None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:00.975142 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-5b07a1dd-5a2f-4bd3-82ce-180472c353da None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:00.975142 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-5b07a1dd-5a2f-4bd3-82ce-180472c353da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:00.975531 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd4e2b-b57c-4925-93c6-b27eaf1ee228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:00.975531 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd4e2b-b57c-4925-93c6-b27eaf1ee228 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.975531 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd4e2b-b57c-4925-93c6-b27eaf1ee228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.975531 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd4e2b-b57c-4925-93c6-b27eaf1ee228 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.975952 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cd4e2b-b57c-4925-93c6-b27eaf1ee228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:00.977527 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cd4e2b-b57c-4925-93c6-b27eaf1ee228 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:00.980928 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 794/1564] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:00.981746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07a1dd-5a2f-4bd3-82ce-180472c353da None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:00.988541 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 771/1565] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:00.989461 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd4e2b-b57c-4925-93c6-b27eaf1ee228 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:00.991120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b07a1dd-5a2f-4bd3-82ce-180472c353da None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:00.996607 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b57e2-9cb6-4113-b4ac-62c4af92767d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:00.997294 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b57e2-9cb6-4113-b4ac-62c4af92767d admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:00.997575 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b57e2-9cb6-4113-b4ac-62c4af92767d admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:00.997850 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b57e2-9cb6-4113-b4ac-62c4af92767d admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:00.998666 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b57e2-9cb6-4113-b4ac-62c4af92767d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:45:00.998979 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b57e2-9cb6-4113-b4ac-62c4af92767d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:00.999393 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd4e2b-b57c-4925-93c6-b27eaf1ee228 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:01.000616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b57e2-9cb6-4113-b4ac-62c4af92767d admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:01.004666 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79611ac0-fd12-4833-944f-49178cdf9b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:01.006519 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79611ac0-fd12-4833-944f-49178cdf9b46 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:01.006519 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79611ac0-fd12-4833-944f-49178cdf9b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:01.006519 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79611ac0-fd12-4833-944f-49178cdf9b46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:01.006772 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79611ac0-fd12-4833-944f-49178cdf9b46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:01.008470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79611ac0-fd12-4833-944f-49178cdf9b46 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:01.021126 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 795/1566] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:45:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:45:01.021126 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b57e2-9cb6-4113-b4ac-62c4af92767d admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:01.032273 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b57e2-9cb6-4113-b4ac-62c4af92767d admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:01.037709 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4aec09-c940-4c75-a48c-2542e1b5c4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:01.037709 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4aec09-c940-4c75-a48c-2542e1b5c4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:01.037709 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4aec09-c940-4c75-a48c-2542e1b5c4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:01.038126 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4aec09-c940-4c75-a48c-2542e1b5c4e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:01.038888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4aec09-c940-4c75-a48c-2542e1b5c4e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:01.040505 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4aec09-c940-4c75-a48c-2542e1b5c4e0 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:01.050557 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 796/1567] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:01.051334 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4aec09-c940-4c75-a48c-2542e1b5c4e0 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:01.060971 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4aec09-c940-4c75-a48c-2542e1b5c4e0 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:01.065473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee8ea1-91d0-420b-aabd-1cd2252faf88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:01.066088 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee8ea1-91d0-420b-aabd-1cd2252faf88 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:01.066372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee8ea1-91d0-420b-aabd-1cd2252faf88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:01.066675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee8ea1-91d0-420b-aabd-1cd2252faf88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:01.067381 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee8ea1-91d0-420b-aabd-1cd2252faf88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:45:01.067688 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee8ea1-91d0-420b-aabd-1cd2252faf88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:01.069376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee8ea1-91d0-420b-aabd-1cd2252faf88 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:01.081177 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 797/1568] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:45:01.081939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee8ea1-91d0-420b-aabd-1cd2252faf88 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:01.091796 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee8ea1-91d0-420b-aabd-1cd2252faf88 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:01.096545 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171e0ab-c809-4e35-a629-31fed3319f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:01.097167 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171e0ab-c809-4e35-a629-31fed3319f87 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:01.097447 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171e0ab-c809-4e35-a629-31fed3319f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:01.097718 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171e0ab-c809-4e35-a629-31fed3319f87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:01.098405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e171e0ab-c809-4e35-a629-31fed3319f87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:01.100317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e171e0ab-c809-4e35-a629-31fed3319f87 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:01.120874 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 772/1569] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:00 2022] POST /identity/v3/projects => generated 387 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:01.121853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79611ac0-fd12-4833-944f-49178cdf9b46 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:01.183668 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79611ac0-fd12-4833-944f-49178cdf9b46 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:01.188146 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8a4e3-828a-4700-83f3-daaaa70dc866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:01.192649 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8a4e3-828a-4700-83f3-daaaa70dc866 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:01.192649 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8a4e3-828a-4700-83f3-daaaa70dc866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:01.192649 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8a4e3-828a-4700-83f3-daaaa70dc866 admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:01.192649 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8a4e3-828a-4700-83f3-daaaa70dc866 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:45:01.192649 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8a4e3-828a-4700-83f3-daaaa70dc866 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:01.192649 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8a4e3-828a-4700-83f3-daaaa70dc866 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:01.203857 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 773/1570] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:45:01.204757 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8a4e3-828a-4700-83f3-daaaa70dc866 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:01.214241 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8a4e3-828a-4700-83f3-daaaa70dc866 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:01.220210 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fb2d3-ab33-461a-a5b0-688e5b5fc387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:01.223281 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 798/1571] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:01 2022] POST /identity/v3/projects => generated 375 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:01.223409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fb2d3-ab33-461a-a5b0-688e5b5fc387 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:01.223524 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171e0ab-c809-4e35-a629-31fed3319f87 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:01.223611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fb2d3-ab33-461a-a5b0-688e5b5fc387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:01.223611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fb2d3-ab33-461a-a5b0-688e5b5fc387 admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:01.223611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0fb2d3-ab33-461a-a5b0-688e5b5fc387 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:45:01.223611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0fb2d3-ab33-461a-a5b0-688e5b5fc387 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:01.223611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0fb2d3-ab33-461a-a5b0-688e5b5fc387 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:01.243003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 774/1572] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:45:01.243003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fb2d3-ab33-461a-a5b0-688e5b5fc387 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:01.248891 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fb2d3-ab33-461a-a5b0-688e5b5fc387 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:01.285400 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171e0ab-c809-4e35-a629-31fed3319f87 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:01.289944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9fbabb-2aeb-4be9-8d7c-85d60f94c62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:01.290903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fbabb-2aeb-4be9-8d7c-85d60f94c62d admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:01.291275 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fbabb-2aeb-4be9-8d7c-85d60f94c62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:01.291553 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fbabb-2aeb-4be9-8d7c-85d60f94c62d admin admin] PATH_INFO: `/v3/projects/87d136c3a8674786a29c2622dfca33e9/users/02865be1eeaa404eabac7612bd86c9d1/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:01.306831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9fbabb-2aeb-4be9-8d7c-85d60f94c62d admin admin] RBAC: Authorizing `identity:create_grant(project_id=87d136c3a8674786a29c2622dfca33e9, user_id=02865be1eeaa404eabac7612bd86c9d1, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:01.308162 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577d539-0307-4cb9-8fde-1b29d15f8009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:01.308496 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: WARNING py.warnings [None req-de9fbabb-2aeb-4be9-8d7c-85d60f94c62d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 18:45:01.308496 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: warnings.warn(msg) Jun 27 18:45:01.308496 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: Jun 27 18:45:01.308818 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577d539-0307-4cb9-8fde-1b29d15f8009 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:01.309098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577d539-0307-4cb9-8fde-1b29d15f8009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:01.309388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577d539-0307-4cb9-8fde-1b29d15f8009 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:01.310060 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3577d539-0307-4cb9-8fde-1b29d15f8009 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:01.310586 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9fbabb-2aeb-4be9-8d7c-85d60f94c62d admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:01.311992 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3577d539-0307-4cb9-8fde-1b29d15f8009 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:01.416538 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 799/1573] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:01 2022] PUT /identity/v3/projects/87d136c3a8674786a29c2622dfca33e9/users/02865be1eeaa404eabac7612bd86c9d1/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:01.417391 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9fbabb-2aeb-4be9-8d7c-85d60f94c62d admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:01.418042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 775/1574] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:01 2022] POST /identity/v3/projects => generated 385 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:01.422299 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577d539-0307-4cb9-8fde-1b29d15f8009 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:01.441107 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:01.481514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9fbabb-2aeb-4be9-8d7c-85d60f94c62d admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:01.484075 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577d539-0307-4cb9-8fde-1b29d15f8009 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:01.486706 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295c5d8-80cd-44ec-8a72-98fd204cfac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:01.487372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295c5d8-80cd-44ec-8a72-98fd204cfac4 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:01.487652 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295c5d8-80cd-44ec-8a72-98fd204cfac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:01.488041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295c5d8-80cd-44ec-8a72-98fd204cfac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:01.488781 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295c5d8-80cd-44ec-8a72-98fd204cfac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:01.489543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c16481-3ade-4419-8c3d-4b873c6e08a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:01.490204 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c16481-3ade-4419-8c3d-4b873c6e08a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:01.490448 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295c5d8-80cd-44ec-8a72-98fd204cfac4 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:01.490646 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c16481-3ade-4419-8c3d-4b873c6e08a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:01.491013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c16481-3ade-4419-8c3d-4b873c6e08a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:01.491895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c16481-3ade-4419-8c3d-4b873c6e08a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:01.494550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c16481-3ade-4419-8c3d-4b873c6e08a4 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:01.503742 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 800/1575] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:01.503742 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295c5d8-80cd-44ec-8a72-98fd204cfac4 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:01.511655 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295c5d8-80cd-44ec-8a72-98fd204cfac4 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:01.516261 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353953f1-c87c-4377-a976-68380a4baed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:01.517016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353953f1-c87c-4377-a976-68380a4baed0 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:01.517326 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353953f1-c87c-4377-a976-68380a4baed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:01.517616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353953f1-c87c-4377-a976-68380a4baed0 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:01.518296 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353953f1-c87c-4377-a976-68380a4baed0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:01.520238 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353953f1-c87c-4377-a976-68380a4baed0 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:01.572674 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 776/1576] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:01 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:01.573635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c16481-3ade-4419-8c3d-4b873c6e08a4 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:01.595311 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:01.600596 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 801/1577] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:01 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:01.607612 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353953f1-c87c-4377-a976-68380a4baed0 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:01.611734 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353953f1-c87c-4377-a976-68380a4baed0 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:01.638587 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c16481-3ade-4419-8c3d-4b873c6e08a4 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:01.644554 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58398df7-9985-4baa-8ba8-c4e5c809cf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:01.645215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58398df7-9985-4baa-8ba8-c4e5c809cf75 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:01.645500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58398df7-9985-4baa-8ba8-c4e5c809cf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:01.645844 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58398df7-9985-4baa-8ba8-c4e5c809cf75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:01.647172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58398df7-9985-4baa-8ba8-c4e5c809cf75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:01.650191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58398df7-9985-4baa-8ba8-c4e5c809cf75 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:01.669688 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad88ef-479b-49cf-9cb8-1c63f1c9c2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:01.670414 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad88ef-479b-49cf-9cb8-1c63f1c9c2da admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:01.670702 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad88ef-479b-49cf-9cb8-1c63f1c9c2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:01.670979 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad88ef-479b-49cf-9cb8-1c63f1c9c2da admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:01.671669 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad88ef-479b-49cf-9cb8-1c63f1c9c2da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:01.673759 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad88ef-479b-49cf-9cb8-1c63f1c9c2da admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:01.789944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 777/1578] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:01 2022] POST /identity/v3/projects => generated 379 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:01.791481 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58398df7-9985-4baa-8ba8-c4e5c809cf75 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:01.820228 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 802/1579] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:01 2022] POST /identity/v3/users => generated 288 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:01.820228 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad88ef-479b-49cf-9cb8-1c63f1c9c2da admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:01.828797 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad88ef-479b-49cf-9cb8-1c63f1c9c2da admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:01.854615 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58398df7-9985-4baa-8ba8-c4e5c809cf75 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:01.859561 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a005ed9-ae9a-43c7-a703-1bd332febc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:01.860215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a005ed9-ae9a-43c7-a703-1bd332febc4f admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:01.860501 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a005ed9-ae9a-43c7-a703-1bd332febc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:01.860782 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a005ed9-ae9a-43c7-a703-1bd332febc4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:01.861636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a005ed9-ae9a-43c7-a703-1bd332febc4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:01.863573 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a005ed9-ae9a-43c7-a703-1bd332febc4f admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:01.887988 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df12f3-e304-4556-bd2a-d97f6a5b7b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:01.892532 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df12f3-e304-4556-bd2a-d97f6a5b7b5f admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:01.892532 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df12f3-e304-4556-bd2a-d97f6a5b7b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:01.892532 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df12f3-e304-4556-bd2a-d97f6a5b7b5f admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:01.892532 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df12f3-e304-4556-bd2a-d97f6a5b7b5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:01.892532 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df12f3-e304-4556-bd2a-d97f6a5b7b5f admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:01.948678 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 778/1580] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:01 2022] POST /identity/v3/projects => generated 405 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:01.950005 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437176d-cb85-4867-b895-b8e6ed767ac3 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:01.950291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437176d-cb85-4867-b895-b8e6ed767ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:01.950599 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437176d-cb85-4867-b895-b8e6ed767ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:01.968891 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:01.976419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-2437176d-cb85-4867-b895-b8e6ed767ac3 None None] MFA Rules not processed for user `02865be1eeaa404eabac7612bd86c9d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:01.979130 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 803/1581] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:01 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:01.979130 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df12f3-e304-4556-bd2a-d97f6a5b7b5f admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:01.979310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-2437176d-cb85-4867-b895-b8e6ed767ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:01.990194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df12f3-e304-4556-bd2a-d97f6a5b7b5f admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:02.044821 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 779/1582] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:02.045645 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437176d-cb85-4867-b895-b8e6ed767ac3 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:02.051119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990e55b-9158-47fa-a337-5e705e3c9d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:02.052029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990e55b-9158-47fa-a337-5e705e3c9d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.052316 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990e55b-9158-47fa-a337-5e705e3c9d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.052646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990e55b-9158-47fa-a337-5e705e3c9d5f admin admin] PATH_INFO: `/v3/projects/0f19df10a9c948ce9c2e0dfa29532e5a/users/2fc6711e48be4d4ab33695e06c7227ab/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.056227 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437176d-cb85-4867-b895-b8e6ed767ac3 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:02.061555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57c457-6533-4532-9383-ef4a7323a85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:02.062992 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57c457-6533-4532-9383-ef4a7323a85c admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.063916 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57c457-6533-4532-9383-ef4a7323a85c admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.064192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57c457-6533-4532-9383-ef4a7323a85c admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.064971 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57c457-6533-4532-9383-ef4a7323a85c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:02.066327 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: WARNING py.warnings [None req-7d57c457-6533-4532-9383-ef4a7323a85c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 18:45:02.066327 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: warnings.warn(msg) Jun 27 18:45:02.066327 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: Jun 27 18:45:02.067674 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57c457-6533-4532-9383-ef4a7323a85c admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:02.069208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4990e55b-9158-47fa-a337-5e705e3c9d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f19df10a9c948ce9c2e0dfa29532e5a, user_id=2fc6711e48be4d4ab33695e06c7227ab, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:02.071961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4990e55b-9158-47fa-a337-5e705e3c9d5f admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:02.155235 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 804/1583] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:01 2022] PUT /identity/v3/projects/0f19df10a9c948ce9c2e0dfa29532e5a/users/2fc6711e48be4d4ab33695e06c7227ab/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:02.156060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990e55b-9158-47fa-a337-5e705e3c9d5f admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:02.159749 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:02.173693 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 780/1584] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:02 2022] POST /identity/v3/users => generated 290 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:02.175360 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57c457-6533-4532-9383-ef4a7323a85c admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:02.185534 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57c457-6533-4532-9383-ef4a7323a85c admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:02.219500 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990e55b-9158-47fa-a337-5e705e3c9d5f admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:02.224008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b5349-1a90-4052-87f9-7be6c4946088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:02.224663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b5349-1a90-4052-87f9-7be6c4946088 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.224942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b5349-1a90-4052-87f9-7be6c4946088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.225217 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b5349-1a90-4052-87f9-7be6c4946088 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.225934 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b5349-1a90-4052-87f9-7be6c4946088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:02.227551 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b5349-1a90-4052-87f9-7be6c4946088 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:02.238927 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 805/1585] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:02.241592 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b5349-1a90-4052-87f9-7be6c4946088 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:02.244499 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d20fe-6f2d-494a-a28a-0fc885b52a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:02.245173 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d20fe-6f2d-494a-a28a-0fc885b52a03 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.245454 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d20fe-6f2d-494a-a28a-0fc885b52a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.245730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d20fe-6f2d-494a-a28a-0fc885b52a03 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.246501 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d20fe-6f2d-494a-a28a-0fc885b52a03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:02.248477 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d20fe-6f2d-494a-a28a-0fc885b52a03 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:02.249697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b5349-1a90-4052-87f9-7be6c4946088 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:02.254149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d1845-bcf7-4c23-94a4-bcebf601f306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:02.254818 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d1845-bcf7-4c23-94a4-bcebf601f306 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.255087 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d1845-bcf7-4c23-94a4-bcebf601f306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.255357 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d1845-bcf7-4c23-94a4-bcebf601f306 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.256068 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d1845-bcf7-4c23-94a4-bcebf601f306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:02.257628 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d1845-bcf7-4c23-94a4-bcebf601f306 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:02.268234 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 806/1586] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:02.268990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d1845-bcf7-4c23-94a4-bcebf601f306 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:02.278631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d1845-bcf7-4c23-94a4-bcebf601f306 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:02.283143 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605c407-cb6b-44e8-8ba3-be1970c84fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:02.287342 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605c407-cb6b-44e8-8ba3-be1970c84fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.287342 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605c407-cb6b-44e8-8ba3-be1970c84fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.287342 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605c407-cb6b-44e8-8ba3-be1970c84fc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.287342 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9605c407-cb6b-44e8-8ba3-be1970c84fc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:02.287342 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9605c407-cb6b-44e8-8ba3-be1970c84fc6 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:02.309811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 781/1587] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:02 2022] POST /identity/v3/users => generated 290 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:02.310651 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d20fe-6f2d-494a-a28a-0fc885b52a03 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:02.320172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d20fe-6f2d-494a-a28a-0fc885b52a03 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:02.325499 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b5458-5ad6-4273-a3d4-42acdb785acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:02.326139 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b5458-5ad6-4273-a3d4-42acdb785acf admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.326453 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b5458-5ad6-4273-a3d4-42acdb785acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.326730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b5458-5ad6-4273-a3d4-42acdb785acf admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.327468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b5458-5ad6-4273-a3d4-42acdb785acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:02.329046 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b5458-5ad6-4273-a3d4-42acdb785acf admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:02.340012 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 782/1588] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:02.341339 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878104fe-2d89-4bf6-98ea-4ef0e8641932 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.341498 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878104fe-2d89-4bf6-98ea-4ef0e8641932 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.341735 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878104fe-2d89-4bf6-98ea-4ef0e8641932 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.342969 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 783/1589] 10.222.0.15 () {54 vars in 961 bytes} [Mon Jun 27 18:45:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:45:02.343993 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82541dcf-36fe-466a-920c-6489753f42b2 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.344284 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82541dcf-36fe-466a-920c-6489753f42b2 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.344539 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82541dcf-36fe-466a-920c-6489753f42b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.362275 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-82541dcf-36fe-466a-920c-6489753f42b2 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:02.363598 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-82541dcf-36fe-466a-920c-6489753f42b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:02.365884 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 807/1590] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:02 2022] POST /identity/v3/users => generated 303 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:02.366845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605c407-cb6b-44e8-8ba3-be1970c84fc6 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:02.373593 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 784/1591] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:02.375317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82541dcf-36fe-466a-920c-6489753f42b2 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:02.379888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605c407-cb6b-44e8-8ba3-be1970c84fc6 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:02.385373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2bf46-1357-47c0-a118-18af52765814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:02.386029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2bf46-1357-47c0-a118-18af52765814 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.386328 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2bf46-1357-47c0-a118-18af52765814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.386637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2bf46-1357-47c0-a118-18af52765814 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.386758 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82541dcf-36fe-466a-920c-6489753f42b2 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:02.387437 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2bf46-1357-47c0-a118-18af52765814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:02.389055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2bf46-1357-47c0-a118-18af52765814 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:02.391403 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4558d-a91c-47e1-8cb5-280c083d21bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:02.392062 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4558d-a91c-47e1-8cb5-280c083d21bd admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.392341 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4558d-a91c-47e1-8cb5-280c083d21bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.392611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4558d-a91c-47e1-8cb5-280c083d21bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.393327 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4558d-a91c-47e1-8cb5-280c083d21bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:02.394944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4558d-a91c-47e1-8cb5-280c083d21bd admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:02.401722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 808/1592] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:02.402680 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2bf46-1357-47c0-a118-18af52765814 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:02.405815 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 785/1593] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:02.406657 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4558d-a91c-47e1-8cb5-280c083d21bd admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:02.413739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2bf46-1357-47c0-a118-18af52765814 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:02.417183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4558d-a91c-47e1-8cb5-280c083d21bd admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:02.420389 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8efc212-14e2-44fc-a136-22c1b3746381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:02.420942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efc212-14e2-44fc-a136-22c1b3746381 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.421274 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efc212-14e2-44fc-a136-22c1b3746381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.421494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efc212-14e2-44fc-a136-22c1b3746381 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.422112 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79692532-04f9-4963-aa08-d82b4528165d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:02.422234 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8efc212-14e2-44fc-a136-22c1b3746381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:02.422945 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79692532-04f9-4963-aa08-d82b4528165d admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.423230 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79692532-04f9-4963-aa08-d82b4528165d admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.423506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79692532-04f9-4963-aa08-d82b4528165d admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.423817 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8efc212-14e2-44fc-a136-22c1b3746381 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:02.424222 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79692532-04f9-4963-aa08-d82b4528165d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:02.425794 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79692532-04f9-4963-aa08-d82b4528165d admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:02.439061 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 809/1594] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:02.439061 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8efc212-14e2-44fc-a136-22c1b3746381 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:02.439316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 786/1595] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:02.439316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f5823-a24f-4134-9747-cb34f02845c3 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.439482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f5823-a24f-4134-9747-cb34f02845c3 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.439678 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f5823-a24f-4134-9747-cb34f02845c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.449794 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8efc212-14e2-44fc-a136-22c1b3746381 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:02.454347 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d089415-ee5d-468e-8803-043b48f59936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:02.455033 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d089415-ee5d-468e-8803-043b48f59936 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.455320 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d089415-ee5d-468e-8803-043b48f59936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.455599 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d089415-ee5d-468e-8803-043b48f59936 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.456329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d089415-ee5d-468e-8803-043b48f59936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:02.457919 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d089415-ee5d-468e-8803-043b48f59936 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:02.458264 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-d58f5823-a24f-4134-9747-cb34f02845c3 None None] MFA Rules not processed for user `1dddd4efb44c4152bb4151f7715f2223`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:02.459479 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-d58f5823-a24f-4134-9747-cb34f02845c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:02.468336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 810/1596] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:02.469071 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d089415-ee5d-468e-8803-043b48f59936 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:02.478964 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d089415-ee5d-468e-8803-043b48f59936 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:02.484051 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3ca41-1cff-46e4-ab34-fd4949b8f8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:02.484693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3ca41-1cff-46e4-ab34-fd4949b8f8e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.484972 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3ca41-1cff-46e4-ab34-fd4949b8f8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.485252 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3ca41-1cff-46e4-ab34-fd4949b8f8e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.485969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb3ca41-1cff-46e4-ab34-fd4949b8f8e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:02.487615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb3ca41-1cff-46e4-ab34-fd4949b8f8e1 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:02.497167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:02.498642 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 811/1597] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:02.499447 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3ca41-1cff-46e4-ab34-fd4949b8f8e1 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:02.536424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 787/1598] 10.222.0.15 () {58 vars in 1058 bytes} [Mon Jun 27 18:45:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:02.537258 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f5823-a24f-4134-9747-cb34f02845c3 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:02.548265 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f5823-a24f-4134-9747-cb34f02845c3 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:02.552782 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516a163-acd0-43e2-82ba-e915d557a3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:02.553437 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516a163-acd0-43e2-82ba-e915d557a3fc admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.553715 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516a163-acd0-43e2-82ba-e915d557a3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.553991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516a163-acd0-43e2-82ba-e915d557a3fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.554741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516a163-acd0-43e2-82ba-e915d557a3fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:02.554929 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3ca41-1cff-46e4-ab34-fd4949b8f8e1 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:02.556433 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516a163-acd0-43e2-82ba-e915d557a3fc admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:02.558663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3908e5-8a3f-4ef1-a38e-c68431f9b384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:02.559292 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3908e5-8a3f-4ef1-a38e-c68431f9b384 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.559579 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3908e5-8a3f-4ef1-a38e-c68431f9b384 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.559903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3908e5-8a3f-4ef1-a38e-c68431f9b384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.570991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 788/1599] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:02.570991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516a163-acd0-43e2-82ba-e915d557a3fc admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:02.571216 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3908e5-8a3f-4ef1-a38e-c68431f9b384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:02.571216 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3908e5-8a3f-4ef1-a38e-c68431f9b384 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:02.578488 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516a163-acd0-43e2-82ba-e915d557a3fc admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:02.580872 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 812/1600] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:02.581622 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3908e5-8a3f-4ef1-a38e-c68431f9b384 service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:02.583399 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4c57e-6bb1-429d-9ab1-8b7f6935ab7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:02.584478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4c57e-6bb1-429d-9ab1-8b7f6935ab7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.584762 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4c57e-6bb1-429d-9ab1-8b7f6935ab7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.585036 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4c57e-6bb1-429d-9ab1-8b7f6935ab7f admin admin] PATH_INFO: `/v3/projects/80ec901270bc4375b22fba22ed2ff68c/users/f6d74ace9f674f0da40cb35f39d559c9/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.592316 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3908e5-8a3f-4ef1-a38e-c68431f9b384 service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:02.596912 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01476b-f9ad-42a4-8ab2-816570e8c56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:02.598064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01476b-f9ad-42a4-8ab2-816570e8c56f admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.598064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01476b-f9ad-42a4-8ab2-816570e8c56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.598333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01476b-f9ad-42a4-8ab2-816570e8c56f admin admin] PATH_INFO: `/v3/projects/bcb48117fbeb47c99a3628629a66d28c/users/3502855d5bf4416997f07bb0cb39710e/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.602845 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e4c57e-6bb1-429d-9ab1-8b7f6935ab7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=80ec901270bc4375b22fba22ed2ff68c, user_id=f6d74ace9f674f0da40cb35f39d559c9, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:02.605723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e4c57e-6bb1-429d-9ab1-8b7f6935ab7f admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:02.614547 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f01476b-f9ad-42a4-8ab2-816570e8c56f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcb48117fbeb47c99a3628629a66d28c, user_id=3502855d5bf4416997f07bb0cb39710e, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:02.617436 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f01476b-f9ad-42a4-8ab2-816570e8c56f admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:02.697981 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 789/1601] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:02 2022] PUT /identity/v3/projects/80ec901270bc4375b22fba22ed2ff68c/users/f6d74ace9f674f0da40cb35f39d559c9/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:02.699299 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4c57e-6bb1-429d-9ab1-8b7f6935ab7f admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:02.720412 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 813/1602] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:02 2022] PUT /identity/v3/projects/bcb48117fbeb47c99a3628629a66d28c/users/3502855d5bf4416997f07bb0cb39710e/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:02.723725 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01476b-f9ad-42a4-8ab2-816570e8c56f admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:02.740598 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:02.784376 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4c57e-6bb1-429d-9ab1-8b7f6935ab7f admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:02.798227 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01476b-f9ad-42a4-8ab2-816570e8c56f admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:02.803421 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f949d-d5dd-4b0b-bd86-4313f14f0e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:02.804297 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f949d-d5dd-4b0b-bd86-4313f14f0e31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.804635 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f949d-d5dd-4b0b-bd86-4313f14f0e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.804938 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f949d-d5dd-4b0b-bd86-4313f14f0e31 admin admin] PATH_INFO: `/v3/projects/be721490bd6c4084a7d13c39a92e0c05/users/e9ce0fe01f5b46b29b9c5a28ca31ac70/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.820724 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f949d-d5dd-4b0b-bd86-4313f14f0e31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be721490bd6c4084a7d13c39a92e0c05, user_id=e9ce0fe01f5b46b29b9c5a28ca31ac70, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:02.824150 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f949d-d5dd-4b0b-bd86-4313f14f0e31 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:02.844266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a432e5a-31f1-4f54-98d5-30d8b7daa7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:02.844937 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a432e5a-31f1-4f54-98d5-30d8b7daa7b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.845217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a432e5a-31f1-4f54-98d5-30d8b7daa7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.845499 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a432e5a-31f1-4f54-98d5-30d8b7daa7b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.846224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a432e5a-31f1-4f54-98d5-30d8b7daa7b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:02.848064 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a432e5a-31f1-4f54-98d5-30d8b7daa7b3 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:02.862564 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 790/1603] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:02.862564 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a432e5a-31f1-4f54-98d5-30d8b7daa7b3 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:02.873196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a432e5a-31f1-4f54-98d5-30d8b7daa7b3 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:02.879092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b03f5-eecc-4603-b3c1-8665eb34ac9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:02.879092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b03f5-eecc-4603-b3c1-8665eb34ac9e admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.879487 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b03f5-eecc-4603-b3c1-8665eb34ac9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.879487 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b03f5-eecc-4603-b3c1-8665eb34ac9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.882950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b03f5-eecc-4603-b3c1-8665eb34ac9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:02.882950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b03f5-eecc-4603-b3c1-8665eb34ac9e admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:02.898940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 791/1604] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:02.899623 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b03f5-eecc-4603-b3c1-8665eb34ac9e admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:02.906971 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 814/1605] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:02 2022] PUT /identity/v3/projects/be721490bd6c4084a7d13c39a92e0c05/users/e9ce0fe01f5b46b29b9c5a28ca31ac70/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:02.907679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f949d-d5dd-4b0b-bd86-4313f14f0e31 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:02.943340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:02.979627 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f949d-d5dd-4b0b-bd86-4313f14f0e31 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:02.984628 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae18b1aa-2c0c-468e-87db-9cbf1da36d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:02.985674 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18b1aa-2c0c-468e-87db-9cbf1da36d51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:02.985958 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18b1aa-2c0c-468e-87db-9cbf1da36d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:02.986239 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18b1aa-2c0c-468e-87db-9cbf1da36d51 admin admin] PATH_INFO: `/v3/projects/f3a70eb47fed4e959fc51400ddc3caf1/users/daef56aadd1c4edfa6eeadea63050d37/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:02.988481 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b03f5-eecc-4603-b3c1-8665eb34ac9e admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:03.001967 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae18b1aa-2c0c-468e-87db-9cbf1da36d51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3a70eb47fed4e959fc51400ddc3caf1, user_id=daef56aadd1c4edfa6eeadea63050d37, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:03.004849 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae18b1aa-2c0c-468e-87db-9cbf1da36d51 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:03.055494 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08496474-a534-4ccc-92c4-91bb484c242a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:03.056825 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08496474-a534-4ccc-92c4-91bb484c242a service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:03.057125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08496474-a534-4ccc-92c4-91bb484c242a service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:03.057495 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08496474-a534-4ccc-92c4-91bb484c242a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:03.101818 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 815/1606] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:02 2022] PUT /identity/v3/projects/f3a70eb47fed4e959fc51400ddc3caf1/users/daef56aadd1c4edfa6eeadea63050d37/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:03.101818 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dea0e4-1ea7-40e9-8311-81ab37d39665 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:03.101818 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dea0e4-1ea7-40e9-8311-81ab37d39665 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:03.101818 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dea0e4-1ea7-40e9-8311-81ab37d39665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:03.124729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-f4dea0e4-1ea7-40e9-8311-81ab37d39665 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:03.126336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-f4dea0e4-1ea7-40e9-8311-81ab37d39665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:03.131820 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08496474-a534-4ccc-92c4-91bb484c242a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:03.135087 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08496474-a534-4ccc-92c4-91bb484c242a service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:03.152234 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 792/1607] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:03.152432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65f5bd-114e-4475-bbc6-99f6a9122e20 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:03.152719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65f5bd-114e-4475-bbc6-99f6a9122e20 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:03.152995 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65f5bd-114e-4475-bbc6-99f6a9122e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:03.184338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-bf65f5bd-114e-4475-bbc6-99f6a9122e20 None None] MFA Rules not processed for user `3502855d5bf4416997f07bb0cb39710e`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:03.186444 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-bf65f5bd-114e-4475-bbc6-99f6a9122e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:03.192232 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 816/1608] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:03.193048 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dea0e4-1ea7-40e9-8311-81ab37d39665 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:03.210169 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dea0e4-1ea7-40e9-8311-81ab37d39665 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:03.215657 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29d617-d0e3-4c63-923c-334b897a2e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:03.217004 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29d617-d0e3-4c63-923c-334b897a2e18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:03.217425 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29d617-d0e3-4c63-923c-334b897a2e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:03.218040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29d617-d0e3-4c63-923c-334b897a2e18 admin admin] PATH_INFO: `/v3/projects/318ef7ddc73e45b98eff7af052f9942f/users/6fd6492e069344abaa9c817b954fecfa/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:03.229543 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:03.235656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:03.239477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc29d617-d0e3-4c63-923c-334b897a2e18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=318ef7ddc73e45b98eff7af052f9942f, user_id=6fd6492e069344abaa9c817b954fecfa, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:03.244470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc29d617-d0e3-4c63-923c-334b897a2e18 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:03.262168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 793/1609] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:03.263234 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65f5bd-114e-4475-bbc6-99f6a9122e20 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:03.275436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65f5bd-114e-4475-bbc6-99f6a9122e20 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:03.281364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49ae7f-3241-4df5-87e8-46e839afb518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:03.282496 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49ae7f-3241-4df5-87e8-46e839afb518 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:03.282859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49ae7f-3241-4df5-87e8-46e839afb518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:03.284182 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49ae7f-3241-4df5-87e8-46e839afb518 admin admin] PATH_INFO: `/v3/projects/b1d1dd35ba3440729652656271622d6d/users/6447e08e8ac342c0a51a74b5c70262a8/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:03.300166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a49ae7f-3241-4df5-87e8-46e839afb518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1d1dd35ba3440729652656271622d6d, user_id=6447e08e8ac342c0a51a74b5c70262a8, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:03.303325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a49ae7f-3241-4df5-87e8-46e839afb518 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:03.327869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 817/1610] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:03 2022] PUT /identity/v3/projects/318ef7ddc73e45b98eff7af052f9942f/users/6fd6492e069344abaa9c817b954fecfa/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:03.329890 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ee1ea-0d20-44dc-a5f8-18bfd1c2e47d None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:03.330337 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ee1ea-0d20-44dc-a5f8-18bfd1c2e47d None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:03.330815 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ee1ea-0d20-44dc-a5f8-18bfd1c2e47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:03.371574 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-487ee1ea-0d20-44dc-a5f8-18bfd1c2e47d None None] MFA Rules not processed for user `e9ce0fe01f5b46b29b9c5a28ca31ac70`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:03.373146 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-487ee1ea-0d20-44dc-a5f8-18bfd1c2e47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:03.381089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 794/1611] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:03 2022] PUT /identity/v3/projects/b1d1dd35ba3440729652656271622d6d/users/6447e08e8ac342c0a51a74b5c70262a8/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:03.382819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4b708-f4bf-47c4-b0fe-16c812174762 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:03.383219 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4b708-f4bf-47c4-b0fe-16c812174762 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:03.383567 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4b708-f4bf-47c4-b0fe-16c812174762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:03.405123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-10d4b708-f4bf-47c4-b0fe-16c812174762 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:03.406493 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-10d4b708-f4bf-47c4-b0fe-16c812174762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:03.446353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 818/1612] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:03.447595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ee1ea-0d20-44dc-a5f8-18bfd1c2e47d None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:03.474632 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 795/1613] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:03.475961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d6fdf-3925-422e-b069-4d0289f756e4 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:03.476078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d6fdf-3925-422e-b069-4d0289f756e4 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:03.476383 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d6fdf-3925-422e-b069-4d0289f756e4 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:03.479013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 796/1614] 10.222.0.15 () {54 vars in 961 bytes} [Mon Jun 27 18:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:45:03.480086 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b6b61-07cd-414a-85f8-0c6b37608449 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:03.480430 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b6b61-07cd-414a-85f8-0c6b37608449 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:03.480778 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b6b61-07cd-414a-85f8-0c6b37608449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:03.511433 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-293b6b61-07cd-414a-85f8-0c6b37608449 None None] MFA Rules not processed for user `6fd6492e069344abaa9c817b954fecfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:03.512643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-293b6b61-07cd-414a-85f8-0c6b37608449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:03.516666 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ee1ea-0d20-44dc-a5f8-18bfd1c2e47d None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:03.521937 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea3fa3-e087-4978-b7c9-4afb32be0141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:03.522672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea3fa3-e087-4978-b7c9-4afb32be0141 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:03.523029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea3fa3-e087-4978-b7c9-4afb32be0141 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:03.523372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea3fa3-e087-4978-b7c9-4afb32be0141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:03.536375 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea3fa3-e087-4978-b7c9-4afb32be0141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:03.539033 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea3fa3-e087-4978-b7c9-4afb32be0141 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:03.554584 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 819/1615] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:03.556500 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd5b60-c459-4245-b3eb-dde0a7cacada None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:03.556855 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd5b60-c459-4245-b3eb-dde0a7cacada None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:03.557094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd5b60-c459-4245-b3eb-dde0a7cacada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:03.575601 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-0cbd5b60-c459-4245-b3eb-dde0a7cacada None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:03.577095 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-0cbd5b60-c459-4245-b3eb-dde0a7cacada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:03.587139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 820/1616] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:03.592154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f904c5d-1817-4f8b-b729-37cc1088df39 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:03.592644 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 797/1617] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:03.593664 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580376c-8685-47fe-90b5-9c615817692e None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:03.593940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580376c-8685-47fe-90b5-9c615817692e None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:03.594218 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580376c-8685-47fe-90b5-9c615817692e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:03.594502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f904c5d-1817-4f8b-b729-37cc1088df39 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:03.594955 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f904c5d-1817-4f8b-b729-37cc1088df39 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:03.596537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 821/1618] 10.222.0.15 () {54 vars in 912 bytes} [Mon Jun 27 18:45:03 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:45:03.597470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f904c5d-1817-4f8b-b729-37cc1088df39 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:03.599593 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-8f904c5d-1817-4f8b-b729-37cc1088df39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:03.613627 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-5580376c-8685-47fe-90b5-9c615817692e None None] MFA Rules not processed for user `1dddd4efb44c4152bb4151f7715f2223`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:03.616137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f904c5d-1817-4f8b-b729-37cc1088df39 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:03.616260 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-5580376c-8685-47fe-90b5-9c615817692e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:03.617495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c37fca-c845-44e7-b3e1-35605d09fa89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:03.618218 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c37fca-c845-44e7-b3e1-35605d09fa89 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:03.619199 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c37fca-c845-44e7-b3e1-35605d09fa89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:03.619591 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c37fca-c845-44e7-b3e1-35605d09fa89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:03.629119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c37fca-c845-44e7-b3e1-35605d09fa89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:03.630662 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c37fca-c845-44e7-b3e1-35605d09fa89 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:03.641298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 822/1619] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:03.644938 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c37fca-c845-44e7-b3e1-35605d09fa89 service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:03.653086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c37fca-c845-44e7-b3e1-35605d09fa89 service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:03.657070 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6b0b3-518f-46b1-a4f2-e9cc0627512c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:03.658216 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6b0b3-518f-46b1-a4f2-e9cc0627512c service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:03.658549 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6b0b3-518f-46b1-a4f2-e9cc0627512c service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:03.659182 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6b0b3-518f-46b1-a4f2-e9cc0627512c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:03.670732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6b0b3-518f-46b1-a4f2-e9cc0627512c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:03.672274 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6b0b3-518f-46b1-a4f2-e9cc0627512c service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:03.675926 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:03.683580 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 823/1620] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:03.684506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6b0b3-518f-46b1-a4f2-e9cc0627512c service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:03.695672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6b0b3-518f-46b1-a4f2-e9cc0627512c service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:03.702554 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a6318-12a0-4efd-8b6a-ce5c9e37b8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:03.703990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a6318-12a0-4efd-8b6a-ce5c9e37b8ab admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:03.704327 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a6318-12a0-4efd-8b6a-ce5c9e37b8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:03.704673 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a6318-12a0-4efd-8b6a-ce5c9e37b8ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:03.705773 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a6318-12a0-4efd-8b6a-ce5c9e37b8ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:03.708009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a6318-12a0-4efd-8b6a-ce5c9e37b8ab admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:03.710690 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 798/1621] 10.222.0.15 () {58 vars in 1058 bytes} [Mon Jun 27 18:45:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:03.713470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de726a-1842-4ca7-8157-e81dc8c63341 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:03.713987 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de726a-1842-4ca7-8157-e81dc8c63341 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:03.719471 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de726a-1842-4ca7-8157-e81dc8c63341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:03.736563 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-36de726a-1842-4ca7-8157-e81dc8c63341 None None] MFA Rules not processed for user `1dddd4efb44c4152bb4151f7715f2223`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:03.737786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-36de726a-1842-4ca7-8157-e81dc8c63341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:03.748333 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 799/1622] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Jun 27 18:45:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:03.749080 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de726a-1842-4ca7-8157-e81dc8c63341 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:03.761701 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de726a-1842-4ca7-8157-e81dc8c63341 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:03.768448 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e6424-6e73-44ed-8fb0-1e7ab45101c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:03.769106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e6424-6e73-44ed-8fb0-1e7ab45101c8 service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:03.769389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e6424-6e73-44ed-8fb0-1e7ab45101c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:03.769667 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e6424-6e73-44ed-8fb0-1e7ab45101c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:03.831124 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e6424-6e73-44ed-8fb0-1e7ab45101c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:03.833559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e6424-6e73-44ed-8fb0-1e7ab45101c8 service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:03.840348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 824/1623] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:03 2022] POST /identity/v3/projects => generated 381 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:03.841204 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a6318-12a0-4efd-8b6a-ce5c9e37b8ab admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:03.857290 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 800/1624] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:03.858509 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e6424-6e73-44ed-8fb0-1e7ab45101c8 service nova] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:03.877413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:03.878250 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e6424-6e73-44ed-8fb0-1e7ab45101c8 service nova] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:03.913272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a6318-12a0-4efd-8b6a-ce5c9e37b8ab admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:03.918125 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a615168-59b7-44d3-beb3-4075ff5e0aa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:03.918999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a615168-59b7-44d3-beb3-4075ff5e0aa9 service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:03.919286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a615168-59b7-44d3-beb3-4075ff5e0aa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:03.919815 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a615168-59b7-44d3-beb3-4075ff5e0aa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:04.031055 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4d6c3-3ba9-4bfa-93ce-191de2d30349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:04.032759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4d6c3-3ba9-4bfa-93ce-191de2d30349 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:04.033094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4d6c3-3ba9-4bfa-93ce-191de2d30349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:04.033379 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4d6c3-3ba9-4bfa-93ce-191de2d30349 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:04.035432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec4d6c3-3ba9-4bfa-93ce-191de2d30349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:04.041173 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec4d6c3-3ba9-4bfa-93ce-191de2d30349 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:04.047550 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a615168-59b7-44d3-beb3-4075ff5e0aa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:04.051428 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a615168-59b7-44d3-beb3-4075ff5e0aa9 service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:04.077910 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 825/1625] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:04.187176 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 801/1626] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:03 2022] POST /identity/v3/users => generated 297 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:04.187833 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a615168-59b7-44d3-beb3-4075ff5e0aa9 service cinder] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:04.196402 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a615168-59b7-44d3-beb3-4075ff5e0aa9 service cinder] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:04.208330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482ebf0-5333-4268-9118-e6a39cdd6b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:04.209254 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482ebf0-5333-4268-9118-e6a39cdd6b94 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:04.209402 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482ebf0-5333-4268-9118-e6a39cdd6b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:04.209587 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482ebf0-5333-4268-9118-e6a39cdd6b94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:04.210294 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482ebf0-5333-4268-9118-e6a39cdd6b94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:04.211961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482ebf0-5333-4268-9118-e6a39cdd6b94 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:04.221890 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4d6c3-3ba9-4bfa-93ce-191de2d30349 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:04.240876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 826/1627] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:04.242339 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4d6c3-3ba9-4bfa-93ce-191de2d30349 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:04.242485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482ebf0-5333-4268-9118-e6a39cdd6b94 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:04.256080 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b581ad-731c-4185-9936-8929baf84365 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:04.256749 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b581ad-731c-4185-9936-8929baf84365 service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:04.257036 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b581ad-731c-4185-9936-8929baf84365 service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:04.258587 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b581ad-731c-4185-9936-8929baf84365 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:04.262526 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482ebf0-5333-4268-9118-e6a39cdd6b94 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:04.273634 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28645558-1ef9-4ba1-a737-8d336d03fdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:04.277080 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28645558-1ef9-4ba1-a737-8d336d03fdff admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:04.277961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28645558-1ef9-4ba1-a737-8d336d03fdff admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:04.280363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28645558-1ef9-4ba1-a737-8d336d03fdff admin admin] PATH_INFO: `/v3/projects/ef2b3b18b54240f19687b89bbe1226b9/users/7f7abeeaef514696bb8868fbb9484d9b/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:04.322001 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28645558-1ef9-4ba1-a737-8d336d03fdff admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef2b3b18b54240f19687b89bbe1226b9, user_id=7f7abeeaef514696bb8868fbb9484d9b, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:04.324945 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28645558-1ef9-4ba1-a737-8d336d03fdff admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:04.391361 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b581ad-731c-4185-9936-8929baf84365 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:04.391361 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b581ad-731c-4185-9936-8929baf84365 service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:04.417317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 802/1628] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:45:04.424104 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 827/1629] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:04 2022] PUT /identity/v3/projects/ef2b3b18b54240f19687b89bbe1226b9/users/7f7abeeaef514696bb8868fbb9484d9b/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:04.428238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b581ad-731c-4185-9936-8929baf84365 service nova] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:04.440497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:04.449132 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b581ad-731c-4185-9936-8929baf84365 service nova] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:04.520801 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b3e99-5126-4ba5-9458-2eb9a346c417 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:04.521404 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b3e99-5126-4ba5-9458-2eb9a346c417 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:04.521698 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b3e99-5126-4ba5-9458-2eb9a346c417 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:04.523613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 828/1630] 10.222.0.15 () {54 vars in 912 bytes} [Mon Jun 27 18:45:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:45:04.529259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b82db2-e54b-4e11-9071-5827a51fb293 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:04.529888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b82db2-e54b-4e11-9071-5827a51fb293 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:04.530343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b82db2-e54b-4e11-9071-5827a51fb293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:04.564099 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-72b82db2-e54b-4e11-9071-5827a51fb293 None None] MFA Rules not processed for user `1dddd4efb44c4152bb4151f7715f2223`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:04.566334 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-72b82db2-e54b-4e11-9071-5827a51fb293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:04.581415 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356a946-0cca-4100-a99b-e8d5bbdade2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:04.582633 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356a946-0cca-4100-a99b-e8d5bbdade2b admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:04.582783 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356a946-0cca-4100-a99b-e8d5bbdade2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:04.583041 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356a946-0cca-4100-a99b-e8d5bbdade2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:04.583893 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356a946-0cca-4100-a99b-e8d5bbdade2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:04.585880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356a946-0cca-4100-a99b-e8d5bbdade2b admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:04.607027 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 803/1631] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:04.611744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356a946-0cca-4100-a99b-e8d5bbdade2b admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:04.623492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356a946-0cca-4100-a99b-e8d5bbdade2b admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:04.629576 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6ae6d-3b01-4eaf-a243-4a94ba35433e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:04.633578 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6ae6d-3b01-4eaf-a243-4a94ba35433e admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:04.633578 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6ae6d-3b01-4eaf-a243-4a94ba35433e admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:04.633578 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6ae6d-3b01-4eaf-a243-4a94ba35433e admin admin] PATH_INFO: `/v3/projects/ef2b3b18b54240f19687b89bbe1226b9/users/7f7abeeaef514696bb8868fbb9484d9b/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:04.636535 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6ae6d-3b01-4eaf-a243-4a94ba35433e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef2b3b18b54240f19687b89bbe1226b9, user_id=7f7abeeaef514696bb8868fbb9484d9b, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:04.640717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6ae6d-3b01-4eaf-a243-4a94ba35433e admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:04.665888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 804/1632] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:04 2022] PUT /identity/v3/projects/ef2b3b18b54240f19687b89bbe1226b9/users/7f7abeeaef514696bb8868fbb9484d9b/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:04.666152 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 829/1633] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Jun 27 18:45:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:45:04.668672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e54b6-8f60-4d55-9ca1-6e5e0dc2fda3 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:04.668961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e54b6-8f60-4d55-9ca1-6e5e0dc2fda3 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:04.669248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e54b6-8f60-4d55-9ca1-6e5e0dc2fda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:04.677764 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b82db2-e54b-4e11-9071-5827a51fb293 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:04.714561 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-e05e54b6-8f60-4d55-9ca1-6e5e0dc2fda3 None None] MFA Rules not processed for user `7f7abeeaef514696bb8868fbb9484d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:04.719154 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-e05e54b6-8f60-4d55-9ca1-6e5e0dc2fda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:04.720577 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:04.761671 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b82db2-e54b-4e11-9071-5827a51fb293 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:04.809173 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 805/1634] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:04 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:04.830397 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a09510-c919-4910-903e-f5b0f37ffbb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:04.831267 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a09510-c919-4910-903e-f5b0f37ffbb6 service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:04.831858 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a09510-c919-4910-903e-f5b0f37ffbb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:04.832374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a09510-c919-4910-903e-f5b0f37ffbb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:04.957598 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a09510-c919-4910-903e-f5b0f37ffbb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:04.962862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a09510-c919-4910-903e-f5b0f37ffbb6 service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:04.986953 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 830/1635] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:45:05.362676 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e54b6-8f60-4d55-9ca1-6e5e0dc2fda3 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:05.374950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e54b6-8f60-4d55-9ca1-6e5e0dc2fda3 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:05.381029 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d0675-f3dc-42fc-8fd9-826dab60f804 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:05.381796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d0675-f3dc-42fc-8fd9-826dab60f804 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:05.382186 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d0675-f3dc-42fc-8fd9-826dab60f804 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:05.382819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d0675-f3dc-42fc-8fd9-826dab60f804 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:05.397215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d0675-f3dc-42fc-8fd9-826dab60f804 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:05.399391 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d0675-f3dc-42fc-8fd9-826dab60f804 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:05.415503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 806/1636] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:05 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:45:06.445522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a09510-c919-4910-903e-f5b0f37ffbb6 service cinder] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:06.551432 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a09510-c919-4910-903e-f5b0f37ffbb6 service cinder] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:06.561497 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e0e85-753c-4395-aef9-732b8a19766f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:06.562876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e0e85-753c-4395-aef9-732b8a19766f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:06.564618 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e0e85-753c-4395-aef9-732b8a19766f admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:06.751465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e0e85-753c-4395-aef9-732b8a19766f admin admin] PATH_INFO: `/v3/users/3502855d5bf4416997f07bb0cb39710e` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:06.751465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e0e85-753c-4395-aef9-732b8a19766f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3502855d5bf4416997f07bb0cb39710e)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:06.751465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: WARNING py.warnings [None req-c16e0e85-753c-4395-aef9-732b8a19766f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 18:45:06.751465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: warnings.warn(msg) Jun 27 18:45:06.751465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: Jun 27 18:45:06.751465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e0e85-753c-4395-aef9-732b8a19766f admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:06.751465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:06.753130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e971b-e75d-4d9f-8234-3ead38bef853 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:06.753130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e971b-e75d-4d9f-8234-3ead38bef853 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:06.753130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e971b-e75d-4d9f-8234-3ead38bef853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:06.753130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-028e971b-e75d-4d9f-8234-3ead38bef853 None None] MFA Rules not processed for user `6447e08e8ac342c0a51a74b5c70262a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:06.753130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-028e971b-e75d-4d9f-8234-3ead38bef853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:06.807215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:06.857286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 807/1637] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:06 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:06.860358 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e971b-e75d-4d9f-8234-3ead38bef853 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:06.888924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-c16e0e85-753c-4395-aef9-732b8a19766f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3502855d5bf4416997f07bb0cb39710e'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:06.889096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e971b-e75d-4d9f-8234-3ead38bef853 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:06.894823 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89faf18-b186-483b-a54b-bb975b14b427 tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f7abeeaef514696bb8868fbb9484d9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef2b3b18b54240f19687b89bbe1226b9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:06.896119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89faf18-b186-483b-a54b-bb975b14b427 tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:06.896945 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89faf18-b186-483b-a54b-bb975b14b427 tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:06.897399 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89faf18-b186-483b-a54b-bb975b14b427 tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:06.898699 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89faf18-b186-483b-a54b-bb975b14b427 tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:45:06.899233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89faf18-b186-483b-a54b-bb975b14b427 tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:06.900937 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-c16e0e85-753c-4395-aef9-732b8a19766f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3502855d5bf4416997f07bb0cb39710e'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:06.901692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-c16e0e85-753c-4395-aef9-732b8a19766f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3502855d5bf4416997f07bb0cb39710e'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:06.908099 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89faf18-b186-483b-a54b-bb975b14b427 tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:06.908317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-c16e0e85-753c-4395-aef9-732b8a19766f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3502855d5bf4416997f07bb0cb39710e'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:06.928880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-c16e0e85-753c-4395-aef9-732b8a19766f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3502855d5bf4416997f07bb0cb39710e'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:06.933434 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 808/1638] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:45:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:45:06.935078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89faf18-b186-483b-a54b-bb975b14b427 tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:06.936959 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-e89faf18-b186-483b-a54b-bb975b14b427 tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:06.952735 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89faf18-b186-483b-a54b-bb975b14b427 tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:06.957814 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c27a7-9c4d-42d2-872c-060312675348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:06.964717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c27a7-9c4d-42d2-872c-060312675348 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:06.964717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c27a7-9c4d-42d2-872c-060312675348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:06.964717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c27a7-9c4d-42d2-872c-060312675348 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:06.964717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c27a7-9c4d-42d2-872c-060312675348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:06.964717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c27a7-9c4d-42d2-872c-060312675348 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:07.022418 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 831/1639] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:45:06 2022] DELETE /identity/v3/users/3502855d5bf4416997f07bb0cb39710e => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:07.022418 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e0e85-753c-4395-aef9-732b8a19766f admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:07.027163 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c16e0e85-753c-4395-aef9-732b8a19766f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:07.106478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 809/1640] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:06 2022] POST /identity/v3/projects => generated 403 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:07.107804 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b2f06-76ec-4de4-a6f2-65708b6509c1 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:07.114036 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b2f06-76ec-4de4-a6f2-65708b6509c1 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:07.114036 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b2f06-76ec-4de4-a6f2-65708b6509c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:07.151592 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-404b2f06-76ec-4de4-a6f2-65708b6509c1 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:07.153696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-404b2f06-76ec-4de4-a6f2-65708b6509c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:07.198247 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e0e85-753c-4395-aef9-732b8a19766f admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:07.276562 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 810/1641] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:07.278780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b2f06-76ec-4de4-a6f2-65708b6509c1 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:07.304493 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b2f06-76ec-4de4-a6f2-65708b6509c1 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:07.308259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f76ab8-57c3-416a-b890-513f50945f7c tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f7abeeaef514696bb8868fbb9484d9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef2b3b18b54240f19687b89bbe1226b9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:07.309351 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f76ab8-57c3-416a-b890-513f50945f7c tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:07.309778 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f76ab8-57c3-416a-b890-513f50945f7c tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:07.310195 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f76ab8-57c3-416a-b890-513f50945f7c tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:07.311208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f76ab8-57c3-416a-b890-513f50945f7c tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:07.313499 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f76ab8-57c3-416a-b890-513f50945f7c tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:07.314997 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6929ee-484f-43f2-abce-3aea260f5ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:07.315947 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6929ee-484f-43f2-abce-3aea260f5ef8 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:07.316290 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6929ee-484f-43f2-abce-3aea260f5ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:07.316777 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6929ee-484f-43f2-abce-3aea260f5ef8 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:07.317857 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6929ee-484f-43f2-abce-3aea260f5ef8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:07.321373 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6929ee-484f-43f2-abce-3aea260f5ef8 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:07.436231 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 832/1642] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:07 2022] POST /identity/v3/projects => generated 353 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:07.437673 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f76ab8-57c3-416a-b890-513f50945f7c tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:07.443309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-24f76ab8-57c3-416a-b890-513f50945f7c tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:07.464162 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 811/1643] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:07 2022] POST /identity/v3/users => generated 308 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:07.467186 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6929ee-484f-43f2-abce-3aea260f5ef8 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:07.492249 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6929ee-484f-43f2-abce-3aea260f5ef8 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:07.546838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:07.592111 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f76ab8-57c3-416a-b890-513f50945f7c tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:07.598738 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79764cea-0eb3-412d-9177-1031d31996a1 tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f7abeeaef514696bb8868fbb9484d9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef2b3b18b54240f19687b89bbe1226b9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:07.602100 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79764cea-0eb3-412d-9177-1031d31996a1 tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:07.604018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392d2c9-0e84-472f-a72a-98d20ee4315b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:07.604381 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392d2c9-0e84-472f-a72a-98d20ee4315b service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:07.604670 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392d2c9-0e84-472f-a72a-98d20ee4315b service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:07.604943 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392d2c9-0e84-472f-a72a-98d20ee4315b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:07.606143 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79764cea-0eb3-412d-9177-1031d31996a1 tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:07.606657 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79764cea-0eb3-412d-9177-1031d31996a1 tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:07.609547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79764cea-0eb3-412d-9177-1031d31996a1 tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:45:07.612763 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79764cea-0eb3-412d-9177-1031d31996a1 tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:07.621332 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79764cea-0eb3-412d-9177-1031d31996a1 tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:07.660405 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 812/1644] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:45:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:45:07.661947 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79764cea-0eb3-412d-9177-1031d31996a1 tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:07.760303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:07.780646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392d2c9-0e84-472f-a72a-98d20ee4315b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:07.782505 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392d2c9-0e84-472f-a72a-98d20ee4315b service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:07.814818 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 833/1645] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:07.815804 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392d2c9-0e84-472f-a72a-98d20ee4315b service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:07.834158 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79764cea-0eb3-412d-9177-1031d31996a1 tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:07.847573 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b85fe-4ba3-49f7-bdb4-6be7960c870e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:07.848701 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b85fe-4ba3-49f7-bdb4-6be7960c870e admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:07.849808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b85fe-4ba3-49f7-bdb4-6be7960c870e admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:07.854173 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b85fe-4ba3-49f7-bdb4-6be7960c870e admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:07.854487 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392d2c9-0e84-472f-a72a-98d20ee4315b service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:07.855813 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b85fe-4ba3-49f7-bdb4-6be7960c870e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:07.861913 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b85fe-4ba3-49f7-bdb4-6be7960c870e admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:07.875744 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755d2a3-1d9d-40bd-8dff-6c7769598aec tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f7abeeaef514696bb8868fbb9484d9b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef2b3b18b54240f19687b89bbe1226b9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:07.877204 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755d2a3-1d9d-40bd-8dff-6c7769598aec tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:07.877688 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755d2a3-1d9d-40bd-8dff-6c7769598aec tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:07.878169 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755d2a3-1d9d-40bd-8dff-6c7769598aec tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] PATH_INFO: `/v3/projects/eb515fe2ea534dc687db5707d595547d` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:07.887818 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6755d2a3-1d9d-40bd-8dff-6c7769598aec tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] RBAC: Authorizing `identity:delete_project(project_id=eb515fe2ea534dc687db5707d595547d)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:07.887818 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6755d2a3-1d9d-40bd-8dff-6c7769598aec tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:07.902282 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 813/1646] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:07.905626 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b85fe-4ba3-49f7-bdb4-6be7960c870e admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:07.919298 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b85fe-4ba3-49f7-bdb4-6be7960c870e admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:07.933686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8db47-e484-4ad3-aa1e-af73bcbeae71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:07.934663 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8db47-e484-4ad3-aa1e-af73bcbeae71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:07.934949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8db47-e484-4ad3-aa1e-af73bcbeae71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:07.935223 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8db47-e484-4ad3-aa1e-af73bcbeae71 admin admin] PATH_INFO: `/v3/projects/468d2ead22fa413b90ca7f9934a11233/users/7be1b248765c4992bc621ce2ab16c0ac/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:07.975837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8db47-e484-4ad3-aa1e-af73bcbeae71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=468d2ead22fa413b90ca7f9934a11233, user_id=7be1b248765c4992bc621ce2ab16c0ac, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:07.982370 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8db47-e484-4ad3-aa1e-af73bcbeae71 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:08.066116 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 814/1647] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:07 2022] PUT /identity/v3/projects/468d2ead22fa413b90ca7f9934a11233/users/7be1b248765c4992bc621ce2ab16c0ac/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:08.069899 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8db47-e484-4ad3-aa1e-af73bcbeae71 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:08.096896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-6755d2a3-1d9d-40bd-8dff-6c7769598aec tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb515fe2ea534dc687db5707d595547d', 'request_id': 'req-6755d2a3-1d9d-40bd-8dff-6c7769598aec'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:08.212164 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-6755d2a3-1d9d-40bd-8dff-6c7769598aec tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb515fe2ea534dc687db5707d595547d', 'request_id': 'req-6755d2a3-1d9d-40bd-8dff-6c7769598aec'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:08.215849 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8db47-e484-4ad3-aa1e-af73bcbeae71 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:08.221353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-6755d2a3-1d9d-40bd-8dff-6c7769598aec tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb515fe2ea534dc687db5707d595547d', 'request_id': 'req-6755d2a3-1d9d-40bd-8dff-6c7769598aec'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:08.245156 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-6755d2a3-1d9d-40bd-8dff-6c7769598aec tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] The token cache is being invalidate because project eb515fe2ea534dc687db5707d595547d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:45:08.247912 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-6755d2a3-1d9d-40bd-8dff-6c7769598aec tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:08.250368 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-6755d2a3-1d9d-40bd-8dff-6c7769598aec tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:08.261853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 834/1648] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:45:07 2022] DELETE /identity/v3/projects/eb515fe2ea534dc687db5707d595547d => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:08.388871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f96d4-89d0-4573-a78f-a360c345168d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:08.389576 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f96d4-89d0-4573-a78f-a360c345168d admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:08.389860 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f96d4-89d0-4573-a78f-a360c345168d admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:08.390145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f96d4-89d0-4573-a78f-a360c345168d admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:08.390936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f96d4-89d0-4573-a78f-a360c345168d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:08.392616 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f96d4-89d0-4573-a78f-a360c345168d admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:08.407888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 815/1649] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:08 2022] GET /identity/v3/roles => generated 1420 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:08.411659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755d2a3-1d9d-40bd-8dff-6c7769598aec tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:08.413109 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-6755d2a3-1d9d-40bd-8dff-6c7769598aec tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:08.452964 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755d2a3-1d9d-40bd-8dff-6c7769598aec tempest-VolumeQuotaClassesTest-1450232364 tempest-VolumeQuotaClassesTest-1450232364-project-admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:08.471052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17d165-f73e-4a96-a855-4670e7cfcc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:08.472553 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17d165-f73e-4a96-a855-4670e7cfcc3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:08.472834 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17d165-f73e-4a96-a855-4670e7cfcc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:08.473119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17d165-f73e-4a96-a855-4670e7cfcc3b admin admin] PATH_INFO: `/v3/projects/468d2ead22fa413b90ca7f9934a11233/users/7be1b248765c4992bc621ce2ab16c0ac/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:08.480407 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd17d165-f73e-4a96-a855-4670e7cfcc3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=468d2ead22fa413b90ca7f9934a11233, user_id=7be1b248765c4992bc621ce2ab16c0ac, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:08.492443 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd17d165-f73e-4a96-a855-4670e7cfcc3b admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:08.513769 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 835/1650] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:08 2022] PUT /identity/v3/projects/468d2ead22fa413b90ca7f9934a11233/users/7be1b248765c4992bc621ce2ab16c0ac/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:08.629532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f96d4-89d0-4573-a78f-a360c345168d admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:08.632715 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-1b0f96d4-89d0-4573-a78f-a360c345168d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:08.652616 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f96d4-89d0-4573-a78f-a360c345168d admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:08.816888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a014fe1-1b58-4d15-9b8a-6a5c829fe42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:08.818642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a014fe1-1b58-4d15-9b8a-6a5c829fe42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:08.819085 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a014fe1-1b58-4d15-9b8a-6a5c829fe42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:08.819500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a014fe1-1b58-4d15-9b8a-6a5c829fe42a admin admin] PATH_INFO: `/v3/users/02865be1eeaa404eabac7612bd86c9d1` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:08.821593 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a014fe1-1b58-4d15-9b8a-6a5c829fe42a admin admin] RBAC: Authorizing `identity:delete_user(user_id=02865be1eeaa404eabac7612bd86c9d1)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:08.824651 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a014fe1-1b58-4d15-9b8a-6a5c829fe42a admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:09.134482 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17d165-f73e-4a96-a855-4670e7cfcc3b admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:09.136797 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-fd17d165-f73e-4a96-a855-4670e7cfcc3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:09.144953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:09.159986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2a014fe1-1b58-4d15-9b8a-6a5c829fe42a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02865be1eeaa404eabac7612bd86c9d1'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:09.183374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2a014fe1-1b58-4d15-9b8a-6a5c829fe42a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02865be1eeaa404eabac7612bd86c9d1'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:09.186832 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2a014fe1-1b58-4d15-9b8a-6a5c829fe42a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02865be1eeaa404eabac7612bd86c9d1'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:09.188390 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17d165-f73e-4a96-a855-4670e7cfcc3b admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:09.205729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a5a76-167e-4253-87fe-8bf695c6f316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:09.206707 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a5a76-167e-4253-87fe-8bf695c6f316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:09.206995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a5a76-167e-4253-87fe-8bf695c6f316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:09.207274 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a5a76-167e-4253-87fe-8bf695c6f316 admin admin] PATH_INFO: `/v3/users/6fd6492e069344abaa9c817b954fecfa` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:09.209418 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a5a76-167e-4253-87fe-8bf695c6f316 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fd6492e069344abaa9c817b954fecfa)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:09.211607 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6a5a76-167e-4253-87fe-8bf695c6f316 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:09.249910 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2a014fe1-1b58-4d15-9b8a-6a5c829fe42a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02865be1eeaa404eabac7612bd86c9d1'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:09.360328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2a014fe1-1b58-4d15-9b8a-6a5c829fe42a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02865be1eeaa404eabac7612bd86c9d1'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:09.368084 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 816/1651] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:45:08 2022] DELETE /identity/v3/users/02865be1eeaa404eabac7612bd86c9d1 => generated 0 bytes in 742 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:09.373166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6af608-8d7d-4c45-8198-9f958052d67c None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:09.373460 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6af608-8d7d-4c45-8198-9f958052d67c None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:09.373742 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6af608-8d7d-4c45-8198-9f958052d67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:09.406575 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-5f6af608-8d7d-4c45-8198-9f958052d67c None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:09.408099 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-5f6af608-8d7d-4c45-8198-9f958052d67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:09.488279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-8c6a5a76-167e-4253-87fe-8bf695c6f316 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fd6492e069344abaa9c817b954fecfa'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:09.520736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-8c6a5a76-167e-4253-87fe-8bf695c6f316 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fd6492e069344abaa9c817b954fecfa'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:09.524229 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-8c6a5a76-167e-4253-87fe-8bf695c6f316 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fd6492e069344abaa9c817b954fecfa'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:09.528171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-8c6a5a76-167e-4253-87fe-8bf695c6f316 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fd6492e069344abaa9c817b954fecfa'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:09.567515 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-8c6a5a76-167e-4253-87fe-8bf695c6f316 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fd6492e069344abaa9c817b954fecfa'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:09.594429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 817/1652] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:45:09.645433 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:09.653206 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6af608-8d7d-4c45-8198-9f958052d67c None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:09.654949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-5f6af608-8d7d-4c45-8198-9f958052d67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:09.684233 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 836/1653] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:45:09 2022] DELETE /identity/v3/users/6fd6492e069344abaa9c817b954fecfa => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:45:09.686023 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f027d97-03c4-40ec-a675-f31d5f059fc4 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:09.686686 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f027d97-03c4-40ec-a675-f31d5f059fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:09.686977 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f027d97-03c4-40ec-a675-f31d5f059fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:09.719316 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-7f027d97-03c4-40ec-a675-f31d5f059fc4 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:09.720559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-7f027d97-03c4-40ec-a675-f31d5f059fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:09.811246 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6af608-8d7d-4c45-8198-9f958052d67c None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:09.949456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 837/1654] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:45:09.950691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f027d97-03c4-40ec-a675-f31d5f059fc4 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:09.958589 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-7f027d97-03c4-40ec-a675-f31d5f059fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:10.001944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475f7c9-406d-42c1-a36e-0f6fa5051830 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:10.003603 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475f7c9-406d-42c1-a36e-0f6fa5051830 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:10.004103 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475f7c9-406d-42c1-a36e-0f6fa5051830 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:10.004594 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475f7c9-406d-42c1-a36e-0f6fa5051830 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:10.010903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f027d97-03c4-40ec-a675-f31d5f059fc4 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:10.034517 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565365af-5a4b-4979-9d63-5a866fb6c8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:10.034517 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565365af-5a4b-4979-9d63-5a866fb6c8c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:10.034517 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565365af-5a4b-4979-9d63-5a866fb6c8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:10.034517 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565365af-5a4b-4979-9d63-5a866fb6c8c1 admin admin] PATH_INFO: `/v3/projects/bcb48117fbeb47c99a3628629a66d28c` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:10.034937 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565365af-5a4b-4979-9d63-5a866fb6c8c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcb48117fbeb47c99a3628629a66d28c)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:10.035380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565365af-5a4b-4979-9d63-5a866fb6c8c1 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:10.035635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475f7c9-406d-42c1-a36e-0f6fa5051830 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:10.037262 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475f7c9-406d-42c1-a36e-0f6fa5051830 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:10.064097 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 818/1655] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:10.087150 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475f7c9-406d-42c1-a36e-0f6fa5051830 service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:10.090511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-5475f7c9-406d-42c1-a36e-0f6fa5051830 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:10.112653 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475f7c9-406d-42c1-a36e-0f6fa5051830 service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:10.117701 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49922bb-044b-4ee1-8c6c-00f20b855bc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:10.120388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49922bb-044b-4ee1-8c6c-00f20b855bc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:10.120875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49922bb-044b-4ee1-8c6c-00f20b855bc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:10.121306 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49922bb-044b-4ee1-8c6c-00f20b855bc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:10.145780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49922bb-044b-4ee1-8c6c-00f20b855bc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:10.148553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49922bb-044b-4ee1-8c6c-00f20b855bc1 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:10.163135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 819/1656] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:45:10.329894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-565365af-5a4b-4979-9d63-5a866fb6c8c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcb48117fbeb47c99a3628629a66d28c', 'request_id': 'req-565365af-5a4b-4979-9d63-5a866fb6c8c1'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:10.441199 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-565365af-5a4b-4979-9d63-5a866fb6c8c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcb48117fbeb47c99a3628629a66d28c', 'request_id': 'req-565365af-5a4b-4979-9d63-5a866fb6c8c1'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:10.455065 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-565365af-5a4b-4979-9d63-5a866fb6c8c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcb48117fbeb47c99a3628629a66d28c', 'request_id': 'req-565365af-5a4b-4979-9d63-5a866fb6c8c1'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:10.499822 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-565365af-5a4b-4979-9d63-5a866fb6c8c1 admin admin] The token cache is being invalidate because project bcb48117fbeb47c99a3628629a66d28c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:45:10.500125 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-565365af-5a4b-4979-9d63-5a866fb6c8c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:10.511067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-565365af-5a4b-4979-9d63-5a866fb6c8c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:10.521232 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 838/1657] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:45:09 2022] DELETE /identity/v3/projects/bcb48117fbeb47c99a3628629a66d28c => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:10.528199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a4320-7776-4d51-9bcc-a71149a1c4e2 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:10.528641 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a4320-7776-4d51-9bcc-a71149a1c4e2 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:10.529026 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2a4320-7776-4d51-9bcc-a71149a1c4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:10.570630 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-6a2a4320-7776-4d51-9bcc-a71149a1c4e2 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:10.572975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-6a2a4320-7776-4d51-9bcc-a71149a1c4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:10.660529 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:10.716085 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 820/1658] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:10.724921 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd3654-7091-4f58-b8a8-1275759d7f66 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:10.725367 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd3654-7091-4f58-b8a8-1275759d7f66 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:10.725968 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd3654-7091-4f58-b8a8-1275759d7f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:10.762404 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-54bd3654-7091-4f58-b8a8-1275759d7f66 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:10.764076 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-54bd3654-7091-4f58-b8a8-1275759d7f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:10.787070 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 839/1659] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:10.791119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a4320-7776-4d51-9bcc-a71149a1c4e2 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:10.804610 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2a4320-7776-4d51-9bcc-a71149a1c4e2 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:10.819101 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69632cb-8bd8-490a-8725-13198f0039b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:10.819937 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69632cb-8bd8-490a-8725-13198f0039b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:10.820278 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69632cb-8bd8-490a-8725-13198f0039b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:10.820659 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69632cb-8bd8-490a-8725-13198f0039b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:10.822066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69632cb-8bd8-490a-8725-13198f0039b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:45:10.822583 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69632cb-8bd8-490a-8725-13198f0039b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:10.825109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69632cb-8bd8-490a-8725-13198f0039b4 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:10.858280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 821/1660] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:45:10.862654 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd3654-7091-4f58-b8a8-1275759d7f66 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:10.873672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd3654-7091-4f58-b8a8-1275759d7f66 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:10.886335 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74524df8-42cb-47e4-851e-aab74f18c936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:10.887169 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74524df8-42cb-47e4-851e-aab74f18c936 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:10.887556 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74524df8-42cb-47e4-851e-aab74f18c936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:10.888710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74524df8-42cb-47e4-851e-aab74f18c936 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:10.889543 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74524df8-42cb-47e4-851e-aab74f18c936 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:10.891710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74524df8-42cb-47e4-851e-aab74f18c936 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:11.045492 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 840/1661] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:10 2022] POST /identity/v3/projects => generated 369 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:11.048869 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69632cb-8bd8-490a-8725-13198f0039b4 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:11.075254 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69632cb-8bd8-490a-8725-13198f0039b4 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:11.199293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d59465-9a2b-44fa-85e0-8f593d3d9ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:11.200382 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d59465-9a2b-44fa-85e0-8f593d3d9ed4 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:11.200831 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d59465-9a2b-44fa-85e0-8f593d3d9ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:11.201255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d59465-9a2b-44fa-85e0-8f593d3d9ed4 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:11.202171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d59465-9a2b-44fa-85e0-8f593d3d9ed4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:11.207956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d59465-9a2b-44fa-85e0-8f593d3d9ed4 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:11.418973 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 822/1662] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:11 2022] POST /identity/v3/users => generated 285 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:11.423109 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74524df8-42cb-47e4-851e-aab74f18c936 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:11.441028 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74524df8-42cb-47e4-851e-aab74f18c936 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:11.459194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f004f1-3177-4d14-8acb-555a806f9d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:11.462689 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f004f1-3177-4d14-8acb-555a806f9d51 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:11.463785 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f004f1-3177-4d14-8acb-555a806f9d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:11.464158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f004f1-3177-4d14-8acb-555a806f9d51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:11.468287 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f004f1-3177-4d14-8acb-555a806f9d51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:11.469970 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f004f1-3177-4d14-8acb-555a806f9d51 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:11.514066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 841/1663] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:45:11.515439 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d59465-9a2b-44fa-85e0-8f593d3d9ed4 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:11.529661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d59465-9a2b-44fa-85e0-8f593d3d9ed4 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:11.540041 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b135db45-03f6-4e07-8269-373d6d9d7da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:11.543012 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135db45-03f6-4e07-8269-373d6d9d7da8 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:11.543311 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135db45-03f6-4e07-8269-373d6d9d7da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:11.545889 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b135db45-03f6-4e07-8269-373d6d9d7da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:11.546142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b135db45-03f6-4e07-8269-373d6d9d7da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:11.556877 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b135db45-03f6-4e07-8269-373d6d9d7da8 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:11.557007 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f004f1-3177-4d14-8acb-555a806f9d51 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:11.557007 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-46f004f1-3177-4d14-8acb-555a806f9d51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:11.589380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f004f1-3177-4d14-8acb-555a806f9d51 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:11.608262 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 823/1664] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:11.609341 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b135db45-03f6-4e07-8269-373d6d9d7da8 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:11.617093 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06409d9d-4e06-486e-a5eb-9b4a05000931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:11.617093 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06409d9d-4e06-486e-a5eb-9b4a05000931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:11.618785 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06409d9d-4e06-486e-a5eb-9b4a05000931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:11.619084 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06409d9d-4e06-486e-a5eb-9b4a05000931 admin admin] PATH_INFO: `/v3/projects/87d136c3a8674786a29c2622dfca33e9` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:11.625727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06409d9d-4e06-486e-a5eb-9b4a05000931 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87d136c3a8674786a29c2622dfca33e9)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:11.629494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06409d9d-4e06-486e-a5eb-9b4a05000931 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:11.640689 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b135db45-03f6-4e07-8269-373d6d9d7da8 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:11.662136 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fcdfac-5ee2-4790-ad37-1c8dd6d05dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:11.663392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fcdfac-5ee2-4790-ad37-1c8dd6d05dcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:11.663632 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fcdfac-5ee2-4790-ad37-1c8dd6d05dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:11.663947 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fcdfac-5ee2-4790-ad37-1c8dd6d05dcb admin admin] PATH_INFO: `/v3/projects/ee377b3bdfe84289b946d3de63caa410/users/2df94bc89c40436abf617814a6d78df3/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:11.713212 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:11.716815 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fcdfac-5ee2-4790-ad37-1c8dd6d05dcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee377b3bdfe84289b946d3de63caa410, user_id=2df94bc89c40436abf617814a6d78df3, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:11.721170 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fcdfac-5ee2-4790-ad37-1c8dd6d05dcb admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:11.847168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 824/1665] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:11 2022] PUT /identity/v3/projects/ee377b3bdfe84289b946d3de63caa410/users/2df94bc89c40436abf617814a6d78df3/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:11.858526 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548c8e4-d428-4a8b-b65e-902a5db8733b None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:11.858851 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548c8e4-d428-4a8b-b65e-902a5db8733b None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:11.859296 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548c8e4-d428-4a8b-b65e-902a5db8733b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:11.861100 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-06409d9d-4e06-486e-a5eb-9b4a05000931 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87d136c3a8674786a29c2622dfca33e9', 'request_id': 'req-06409d9d-4e06-486e-a5eb-9b4a05000931'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:11.910293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-2548c8e4-d428-4a8b-b65e-902a5db8733b None None] MFA Rules not processed for user `2df94bc89c40436abf617814a6d78df3`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:11.912209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-2548c8e4-d428-4a8b-b65e-902a5db8733b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:11.947612 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-06409d9d-4e06-486e-a5eb-9b4a05000931 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87d136c3a8674786a29c2622dfca33e9', 'request_id': 'req-06409d9d-4e06-486e-a5eb-9b4a05000931'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:11.964068 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-06409d9d-4e06-486e-a5eb-9b4a05000931 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87d136c3a8674786a29c2622dfca33e9', 'request_id': 'req-06409d9d-4e06-486e-a5eb-9b4a05000931'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:11.981022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-06409d9d-4e06-486e-a5eb-9b4a05000931 admin admin] The token cache is being invalidate because project 87d136c3a8674786a29c2622dfca33e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:45:11.981320 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-06409d9d-4e06-486e-a5eb-9b4a05000931 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:11.982007 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-06409d9d-4e06-486e-a5eb-9b4a05000931 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:11.983866 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 842/1666] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:45:11 2022] DELETE /identity/v3/projects/87d136c3a8674786a29c2622dfca33e9 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:45:11.995042 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06409d9d-4e06-486e-a5eb-9b4a05000931 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:11.996260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-06409d9d-4e06-486e-a5eb-9b4a05000931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:12.106317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 825/1667] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:11 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:45:12.107815 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c0a79-2578-415d-b657-82acada3732b None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:12.108493 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c0a79-2578-415d-b657-82acada3732b None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:12.108776 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c0a79-2578-415d-b657-82acada3732b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:12.136584 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-c60c0a79-2578-415d-b657-82acada3732b None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:12.137837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-c60c0a79-2578-415d-b657-82acada3732b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:12.171466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06409d9d-4e06-486e-a5eb-9b4a05000931 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:12.266907 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 826/1668] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:12.266907 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c0a79-2578-415d-b657-82acada3732b None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:12.266907 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-c60c0a79-2578-415d-b657-82acada3732b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:12.277778 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c0a79-2578-415d-b657-82acada3732b None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:12.283206 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc2d3b-3c1e-4545-96c3-5c2ea12dda7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:12.283611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc2d3b-3c1e-4545-96c3-5c2ea12dda7d admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:12.283968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc2d3b-3c1e-4545-96c3-5c2ea12dda7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:12.284315 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc2d3b-3c1e-4545-96c3-5c2ea12dda7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:12.285091 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc2d3b-3c1e-4545-96c3-5c2ea12dda7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:12.287387 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc2d3b-3c1e-4545-96c3-5c2ea12dda7d admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:12.312538 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0a9e6-f99e-4d46-b9d1-247e35c6b241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:12.314577 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0a9e6-f99e-4d46-b9d1-247e35c6b241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:12.315118 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0a9e6-f99e-4d46-b9d1-247e35c6b241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:12.317907 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0a9e6-f99e-4d46-b9d1-247e35c6b241 admin admin] PATH_INFO: `/v3/projects/318ef7ddc73e45b98eff7af052f9942f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:12.320422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0a9e6-f99e-4d46-b9d1-247e35c6b241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=318ef7ddc73e45b98eff7af052f9942f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:12.324408 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0a9e6-f99e-4d46-b9d1-247e35c6b241 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:12.468463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 827/1669] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:12 2022] POST /identity/v3/projects => generated 369 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:12.468463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c932e-aa9b-4707-a57c-bc9339b149f2 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:12.468463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c932e-aa9b-4707-a57c-bc9339b149f2 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:12.468463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c932e-aa9b-4707-a57c-bc9339b149f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:12.513045 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-388c932e-aa9b-4707-a57c-bc9339b149f2 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:12.515107 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-388c932e-aa9b-4707-a57c-bc9339b149f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:12.555353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b1b0a9e6-f99e-4d46-b9d1-247e35c6b241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '318ef7ddc73e45b98eff7af052f9942f', 'request_id': 'req-b1b0a9e6-f99e-4d46-b9d1-247e35c6b241'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:12.559191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:12.650864 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b1b0a9e6-f99e-4d46-b9d1-247e35c6b241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '318ef7ddc73e45b98eff7af052f9942f', 'request_id': 'req-b1b0a9e6-f99e-4d46-b9d1-247e35c6b241'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:12.657807 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b1b0a9e6-f99e-4d46-b9d1-247e35c6b241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '318ef7ddc73e45b98eff7af052f9942f', 'request_id': 'req-b1b0a9e6-f99e-4d46-b9d1-247e35c6b241'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:12.680431 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b1b0a9e6-f99e-4d46-b9d1-247e35c6b241 admin admin] The token cache is being invalidate because project 318ef7ddc73e45b98eff7af052f9942f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:45:12.680861 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b1b0a9e6-f99e-4d46-b9d1-247e35c6b241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:12.681767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b1b0a9e6-f99e-4d46-b9d1-247e35c6b241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:12.686590 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 843/1670] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:45:11 2022] DELETE /identity/v3/projects/318ef7ddc73e45b98eff7af052f9942f => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:12.688008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7ae61-953e-422f-bc44-1076be6c0eda None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:12.688302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7ae61-953e-422f-bc44-1076be6c0eda None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:12.688588 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7ae61-953e-422f-bc44-1076be6c0eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:12.694945 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 828/1671] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:12.705600 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c932e-aa9b-4707-a57c-bc9339b149f2 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:12.737342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c932e-aa9b-4707-a57c-bc9339b149f2 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:12.751610 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aac73d-2390-4196-b29e-d13a2bbe5d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:12.752612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aac73d-2390-4196-b29e-d13a2bbe5d7e admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:12.752955 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aac73d-2390-4196-b29e-d13a2bbe5d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:12.753232 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aac73d-2390-4196-b29e-d13a2bbe5d7e admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:12.754027 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aac73d-2390-4196-b29e-d13a2bbe5d7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:12.759895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aac73d-2390-4196-b29e-d13a2bbe5d7e admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:12.765018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-6ad7ae61-953e-422f-bc44-1076be6c0eda None None] MFA Rules not processed for user `7be1b248765c4992bc621ce2ab16c0ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:12.765018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-6ad7ae61-953e-422f-bc44-1076be6c0eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:12.855006 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 829/1672] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:12 2022] POST /identity/v3/users => generated 291 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:12.860643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aac73d-2390-4196-b29e-d13a2bbe5d7e admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:12.866688 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-00aac73d-2390-4196-b29e-d13a2bbe5d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:12.908731 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aac73d-2390-4196-b29e-d13a2bbe5d7e admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:12.934428 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7033b-79fd-4cc0-af2c-5a7cb6b68b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:12.935798 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7033b-79fd-4cc0-af2c-5a7cb6b68b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:12.935798 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7033b-79fd-4cc0-af2c-5a7cb6b68b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:12.936009 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7033b-79fd-4cc0-af2c-5a7cb6b68b9e admin admin] PATH_INFO: `/v3/users/7f7abeeaef514696bb8868fbb9484d9b` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:12.954446 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 844/1673] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:12 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:12.960606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7ae61-953e-422f-bc44-1076be6c0eda None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:12.966529 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f7033b-79fd-4cc0-af2c-5a7cb6b68b9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f7abeeaef514696bb8868fbb9484d9b)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:12.967491 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-6ad7ae61-953e-422f-bc44-1076be6c0eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:12.972452 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f7033b-79fd-4cc0-af2c-5a7cb6b68b9e admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:13.031910 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7ae61-953e-422f-bc44-1076be6c0eda None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:13.040669 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5018fc-e939-4d32-8386-a6991c276350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:13.041716 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5018fc-e939-4d32-8386-a6991c276350 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:13.042158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5018fc-e939-4d32-8386-a6991c276350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:13.042625 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5018fc-e939-4d32-8386-a6991c276350 admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:13.043562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5018fc-e939-4d32-8386-a6991c276350 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:45:13.044028 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5018fc-e939-4d32-8386-a6991c276350 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:13.047597 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5018fc-e939-4d32-8386-a6991c276350 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:13.084674 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 845/1674] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:45:13.084674 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5018fc-e939-4d32-8386-a6991c276350 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:13.085914 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ad5018fc-e939-4d32-8386-a6991c276350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:13.113739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5018fc-e939-4d32-8386-a6991c276350 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:13.127024 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a0eed-6ac9-47a2-a5c7-b17232d04828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:13.127890 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a0eed-6ac9-47a2-a5c7-b17232d04828 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:13.128348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a0eed-6ac9-47a2-a5c7-b17232d04828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:13.128770 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a0eed-6ac9-47a2-a5c7-b17232d04828 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:13.130272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a0eed-6ac9-47a2-a5c7-b17232d04828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:13.139550 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a0eed-6ac9-47a2-a5c7-b17232d04828 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:13.159781 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 846/1675] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:13.161071 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a0eed-6ac9-47a2-a5c7-b17232d04828 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:13.189411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a0eed-6ac9-47a2-a5c7-b17232d04828 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:13.203504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d852787-9dd2-4ddb-962d-19bddbf321e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:13.204382 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d852787-9dd2-4ddb-962d-19bddbf321e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:13.204479 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d852787-9dd2-4ddb-962d-19bddbf321e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:13.204738 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d852787-9dd2-4ddb-962d-19bddbf321e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:13.207177 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d852787-9dd2-4ddb-962d-19bddbf321e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:13.215311 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d852787-9dd2-4ddb-962d-19bddbf321e1 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:13.256151 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-64f7033b-79fd-4cc0-af2c-5a7cb6b68b9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f7abeeaef514696bb8868fbb9484d9b'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:13.275029 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-64f7033b-79fd-4cc0-af2c-5a7cb6b68b9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f7abeeaef514696bb8868fbb9484d9b'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:13.276334 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-64f7033b-79fd-4cc0-af2c-5a7cb6b68b9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f7abeeaef514696bb8868fbb9484d9b'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:13.331913 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-64f7033b-79fd-4cc0-af2c-5a7cb6b68b9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f7abeeaef514696bb8868fbb9484d9b'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:13.370682 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 847/1676] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:13 2022] POST /identity/v3/projects => generated 373 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:13.370682 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d852787-9dd2-4ddb-962d-19bddbf321e1 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:13.445097 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-64f7033b-79fd-4cc0-af2c-5a7cb6b68b9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f7abeeaef514696bb8868fbb9484d9b'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:13.460323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 830/1677] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:45:12 2022] DELETE /identity/v3/users/7f7abeeaef514696bb8868fbb9484d9b => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:13.464730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7033b-79fd-4cc0-af2c-5a7cb6b68b9e admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:13.466210 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-64f7033b-79fd-4cc0-af2c-5a7cb6b68b9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:13.508040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d852787-9dd2-4ddb-962d-19bddbf321e1 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:13.571874 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:13.580645 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8fd40a-d78c-4a90-a97b-5939b2638a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:13.581947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8fd40a-d78c-4a90-a97b-5939b2638a9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:13.582373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8fd40a-d78c-4a90-a97b-5939b2638a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:13.582855 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8fd40a-d78c-4a90-a97b-5939b2638a9f admin admin] PATH_INFO: `/v3/projects/f239ff88533040a59e227992c5538480/users/987f7e12cd294ddabd634eee97c99b33/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:13.600792 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:13.605141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8fd40a-d78c-4a90-a97b-5939b2638a9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f239ff88533040a59e227992c5538480, user_id=987f7e12cd294ddabd634eee97c99b33, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:13.615951 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7033b-79fd-4cc0-af2c-5a7cb6b68b9e admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:13.616104 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8fd40a-d78c-4a90-a97b-5939b2638a9f admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:13.623077 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b068a2c-5c4f-4408-bc8e-688199b27358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:13.623790 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b068a2c-5c4f-4408-bc8e-688199b27358 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:13.624079 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b068a2c-5c4f-4408-bc8e-688199b27358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:13.624356 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b068a2c-5c4f-4408-bc8e-688199b27358 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:13.625120 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b068a2c-5c4f-4408-bc8e-688199b27358 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:13.627151 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b068a2c-5c4f-4408-bc8e-688199b27358 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:13.692535 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 848/1678] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:13 2022] PUT /identity/v3/projects/f239ff88533040a59e227992c5538480/users/987f7e12cd294ddabd634eee97c99b33/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:13.696731 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8fd40a-d78c-4a90-a97b-5939b2638a9f admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:13.701656 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-9f8fd40a-d78c-4a90-a97b-5939b2638a9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:13.730085 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 831/1679] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:13 2022] POST /identity/v3/users => generated 287 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:13.730085 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e460999-76bc-412d-84ba-c8964a2298d7 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:13.730085 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e460999-76bc-412d-84ba-c8964a2298d7 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:13.730085 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e460999-76bc-412d-84ba-c8964a2298d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:13.787911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-9e460999-76bc-412d-84ba-c8964a2298d7 None None] MFA Rules not processed for user `2fc6711e48be4d4ab33695e06c7227ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:13.789812 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-9e460999-76bc-412d-84ba-c8964a2298d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:13.809608 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8fd40a-d78c-4a90-a97b-5939b2638a9f admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:13.815102 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787930d-0c3a-44b2-a87d-f9ace75cae34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:13.816172 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787930d-0c3a-44b2-a87d-f9ace75cae34 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:13.816611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787930d-0c3a-44b2-a87d-f9ace75cae34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:13.817031 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787930d-0c3a-44b2-a87d-f9ace75cae34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:13.817923 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1787930d-0c3a-44b2-a87d-f9ace75cae34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:13.820617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1787930d-0c3a-44b2-a87d-f9ace75cae34 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:13.844434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 849/1680] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:45:13.848607 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787930d-0c3a-44b2-a87d-f9ace75cae34 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:13.871819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787930d-0c3a-44b2-a87d-f9ace75cae34 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:13.886242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0615f9-7419-4fca-8258-5e547ef2a6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:13.886242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0615f9-7419-4fca-8258-5e547ef2a6b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:13.886242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0615f9-7419-4fca-8258-5e547ef2a6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:13.886242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0615f9-7419-4fca-8258-5e547ef2a6b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:13.886575 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0615f9-7419-4fca-8258-5e547ef2a6b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:13.886575 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0615f9-7419-4fca-8258-5e547ef2a6b5 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:13.904508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 850/1681] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:13.904508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0615f9-7419-4fca-8258-5e547ef2a6b5 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:13.915629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0615f9-7419-4fca-8258-5e547ef2a6b5 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:13.924857 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bcd2d7-0b9d-4f96-9884-a728048d2f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:13.925776 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcd2d7-0b9d-4f96-9884-a728048d2f9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:13.926064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcd2d7-0b9d-4f96-9884-a728048d2f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:13.926430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcd2d7-0b9d-4f96-9884-a728048d2f9c admin admin] PATH_INFO: `/v3/projects/f239ff88533040a59e227992c5538480/users/987f7e12cd294ddabd634eee97c99b33/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:13.928782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bcd2d7-0b9d-4f96-9884-a728048d2f9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f239ff88533040a59e227992c5538480, user_id=987f7e12cd294ddabd634eee97c99b33, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:13.932443 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bcd2d7-0b9d-4f96-9884-a728048d2f9c admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:13.964777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 851/1682] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:13 2022] PUT /identity/v3/projects/f239ff88533040a59e227992c5538480/users/987f7e12cd294ddabd634eee97c99b33/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:13.971148 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bcd2d7-0b9d-4f96-9884-a728048d2f9c admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:13.980447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 832/1683] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:45:13.982764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470521c6-45af-4eb6-b152-48b1c5968d3c None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:13.983897 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470521c6-45af-4eb6-b152-48b1c5968d3c None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:13.984340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470521c6-45af-4eb6-b152-48b1c5968d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:14.021711 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-470521c6-45af-4eb6-b152-48b1c5968d3c None None] MFA Rules not processed for user `987f7e12cd294ddabd634eee97c99b33`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:14.023254 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-470521c6-45af-4eb6-b152-48b1c5968d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:14.148058 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 833/1684] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:13 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:14.153118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470521c6-45af-4eb6-b152-48b1c5968d3c None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:14.153118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-470521c6-45af-4eb6-b152-48b1c5968d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:14.206898 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bcd2d7-0b9d-4f96-9884-a728048d2f9c admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:14.217978 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0a696-1189-4989-954c-763f08cfbd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:14.218747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0a696-1189-4989-954c-763f08cfbd50 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:14.219341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0a696-1189-4989-954c-763f08cfbd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:14.219448 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0a696-1189-4989-954c-763f08cfbd50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:14.220477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e0a696-1189-4989-954c-763f08cfbd50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:14.221966 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e0a696-1189-4989-954c-763f08cfbd50 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:14.246649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 852/1685] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:14.247766 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0a696-1189-4989-954c-763f08cfbd50 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:14.251853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-35e0a696-1189-4989-954c-763f08cfbd50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:14.326877 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470521c6-45af-4eb6-b152-48b1c5968d3c None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:14.344581 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b16962-19d7-4763-bbb0-516c72d6075f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:14.345542 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b16962-19d7-4763-bbb0-516c72d6075f service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:14.345787 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b16962-19d7-4763-bbb0-516c72d6075f service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:14.346235 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b16962-19d7-4763-bbb0-516c72d6075f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:14.372242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0a696-1189-4989-954c-763f08cfbd50 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:14.376362 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107c556-6684-4b06-8fdf-726ebe782ca9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:14.376916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107c556-6684-4b06-8fdf-726ebe782ca9 service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:14.377198 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107c556-6684-4b06-8fdf-726ebe782ca9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:14.377473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107c556-6684-4b06-8fdf-726ebe782ca9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:14.389978 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107c556-6684-4b06-8fdf-726ebe782ca9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:14.391732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107c556-6684-4b06-8fdf-726ebe782ca9 service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:14.418918 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 853/1686] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:14 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:14.421287 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107c556-6684-4b06-8fdf-726ebe782ca9 service cinder] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:14.453963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107c556-6684-4b06-8fdf-726ebe782ca9 service cinder] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:14.470124 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03033751-b90e-41df-83c5-67d94fb73737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:14.471280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03033751-b90e-41df-83c5-67d94fb73737 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:14.471665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03033751-b90e-41df-83c5-67d94fb73737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:14.472045 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03033751-b90e-41df-83c5-67d94fb73737 admin admin] PATH_INFO: `/v3/projects/09af9f0756c84b5787f9a184b3aff8a3/users/deeed099bc2b40d1b43f6ef1a03d0541/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:14.498896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03033751-b90e-41df-83c5-67d94fb73737 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09af9f0756c84b5787f9a184b3aff8a3, user_id=deeed099bc2b40d1b43f6ef1a03d0541, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:14.512837 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03033751-b90e-41df-83c5-67d94fb73737 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:14.522971 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b16962-19d7-4763-bbb0-516c72d6075f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:14.526974 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b16962-19d7-4763-bbb0-516c72d6075f service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:14.544747 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 834/1687] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:14.578234 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b16962-19d7-4763-bbb0-516c72d6075f service nova] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:14.590850 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b16962-19d7-4763-bbb0-516c72d6075f service nova] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:14.604412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18452e6c-15c0-4772-96b4-092df5a91fda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:14.605197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18452e6c-15c0-4772-96b4-092df5a91fda service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:14.612596 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18452e6c-15c0-4772-96b4-092df5a91fda service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:14.612596 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18452e6c-15c0-4772-96b4-092df5a91fda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:14.612596 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-18452e6c-15c0-4772-96b4-092df5a91fda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:14.640105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 854/1688] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:14 2022] PUT /identity/v3/projects/09af9f0756c84b5787f9a184b3aff8a3/users/deeed099bc2b40d1b43f6ef1a03d0541/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:14.652654 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d684337-cea5-4e07-b4df-35425a17f122 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:14.653137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d684337-cea5-4e07-b4df-35425a17f122 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:14.653585 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d684337-cea5-4e07-b4df-35425a17f122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:14.704527 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=83,keystone:INSERT=7,keystone:DELETE=10 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:14.721971 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-1d684337-cea5-4e07-b4df-35425a17f122 None None] MFA Rules not processed for user `deeed099bc2b40d1b43f6ef1a03d0541`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:14.721971 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-1d684337-cea5-4e07-b4df-35425a17f122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:14.800932 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18452e6c-15c0-4772-96b4-092df5a91fda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:14.803031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18452e6c-15c0-4772-96b4-092df5a91fda service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:14.838447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 835/1689] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:14 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:45:14.864652 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 855/1690] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:45:14.870046 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18452e6c-15c0-4772-96b4-092df5a91fda service cinder] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:14.890328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18452e6c-15c0-4772-96b4-092df5a91fda service cinder] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:15.034173 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:15.035225 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375e97f-c97f-48e1-bc87-198beb085e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:15.035900 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375e97f-c97f-48e1-bc87-198beb085e05 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:15.036297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375e97f-c97f-48e1-bc87-198beb085e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:15.036680 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375e97f-c97f-48e1-bc87-198beb085e05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:15.037492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375e97f-c97f-48e1-bc87-198beb085e05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:15.043941 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375e97f-c97f-48e1-bc87-198beb085e05 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:15.193250 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 836/1691] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:14 2022] POST /identity/v3/projects => generated 373 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:15.195660 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d684337-cea5-4e07-b4df-35425a17f122 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:15.229639 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d684337-cea5-4e07-b4df-35425a17f122 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:15.304375 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1b51b-f037-4777-99ec-7d37c4ac5ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:15.306076 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1b51b-f037-4777-99ec-7d37c4ac5ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:15.306816 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1b51b-f037-4777-99ec-7d37c4ac5ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:15.307351 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1b51b-f037-4777-99ec-7d37c4ac5ed7 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:15.308249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1b51b-f037-4777-99ec-7d37c4ac5ed7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:15.311463 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1b51b-f037-4777-99ec-7d37c4ac5ed7 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:15.420198 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 856/1692] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:15 2022] POST /identity/v3/users => generated 293 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:15.424010 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375e97f-c97f-48e1-bc87-198beb085e05 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:15.448895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375e97f-c97f-48e1-bc87-198beb085e05 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:15.461344 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064b5d4-f884-4c6c-8981-0f50a46d1598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:15.462654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064b5d4-f884-4c6c-8981-0f50a46d1598 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:15.463228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064b5d4-f884-4c6c-8981-0f50a46d1598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:15.463228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064b5d4-f884-4c6c-8981-0f50a46d1598 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:15.464369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064b5d4-f884-4c6c-8981-0f50a46d1598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:15.467958 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064b5d4-f884-4c6c-8981-0f50a46d1598 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:15.487098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 837/1693] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:45:15.491289 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1b51b-f037-4777-99ec-7d37c4ac5ed7 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:15.508553 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1b51b-f037-4777-99ec-7d37c4ac5ed7 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:15.517021 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dda39e-e097-47c8-8e44-041d29c0b30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:15.522916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dda39e-e097-47c8-8e44-041d29c0b30d admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:15.523479 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dda39e-e097-47c8-8e44-041d29c0b30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:15.523970 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dda39e-e097-47c8-8e44-041d29c0b30d admin admin] PATH_INFO: `/v3/projects/5212908648524c769045482fe4d40c69/users/ca102d4720334b548b089adb37f0cf91/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:15.579349 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dda39e-e097-47c8-8e44-041d29c0b30d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5212908648524c769045482fe4d40c69, user_id=ca102d4720334b548b089adb37f0cf91, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:15.582692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dda39e-e097-47c8-8e44-041d29c0b30d admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:15.683592 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 857/1694] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:15 2022] PUT /identity/v3/projects/5212908648524c769045482fe4d40c69/users/ca102d4720334b548b089adb37f0cf91/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:15.684329 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064b5d4-f884-4c6c-8981-0f50a46d1598 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:15.718735 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064b5d4-f884-4c6c-8981-0f50a46d1598 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:15.719902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dda39e-e097-47c8-8e44-041d29c0b30d admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:15.722030 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c8dda39e-e097-47c8-8e44-041d29c0b30d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:15.910913 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390f77d-dbbd-435a-b4e5-95ea88140d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:15.912015 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390f77d-dbbd-435a-b4e5-95ea88140d66 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:15.912528 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390f77d-dbbd-435a-b4e5-95ea88140d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:15.913025 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390f77d-dbbd-435a-b4e5-95ea88140d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:15.914542 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390f77d-dbbd-435a-b4e5-95ea88140d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:15.917111 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390f77d-dbbd-435a-b4e5-95ea88140d66 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:15.919431 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dda39e-e097-47c8-8e44-041d29c0b30d admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:15.935104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 838/1695] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:15.935104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390f77d-dbbd-435a-b4e5-95ea88140d66 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:15.937899 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaca87d-c594-47d1-96a4-dbe4f3d0db64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:15.937899 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaca87d-c594-47d1-96a4-dbe4f3d0db64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:15.937899 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaca87d-c594-47d1-96a4-dbe4f3d0db64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:15.939244 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaca87d-c594-47d1-96a4-dbe4f3d0db64 admin admin] PATH_INFO: `/v3/projects/ef2b3b18b54240f19687b89bbe1226b9` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:15.942049 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaca87d-c594-47d1-96a4-dbe4f3d0db64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef2b3b18b54240f19687b89bbe1226b9)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:15.950975 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaca87d-c594-47d1-96a4-dbe4f3d0db64 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:15.970912 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-2390f77d-dbbd-435a-b4e5-95ea88140d66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:15.992152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390f77d-dbbd-435a-b4e5-95ea88140d66 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:15.999154 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5810226d-17b1-4982-88e1-d0a694ee3195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:16.003368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810226d-17b1-4982-88e1-d0a694ee3195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:16.003838 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810226d-17b1-4982-88e1-d0a694ee3195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:16.004270 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810226d-17b1-4982-88e1-d0a694ee3195 admin admin] PATH_INFO: `/v3/users/e9ce0fe01f5b46b29b9c5a28ca31ac70` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:16.006313 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810226d-17b1-4982-88e1-d0a694ee3195 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9ce0fe01f5b46b29b9c5a28ca31ac70)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:16.009836 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810226d-17b1-4982-88e1-d0a694ee3195 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:16.184874 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-fdaca87d-c594-47d1-96a4-dbe4f3d0db64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef2b3b18b54240f19687b89bbe1226b9', 'request_id': 'req-fdaca87d-c594-47d1-96a4-dbe4f3d0db64'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:16.275243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-fdaca87d-c594-47d1-96a4-dbe4f3d0db64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef2b3b18b54240f19687b89bbe1226b9', 'request_id': 'req-fdaca87d-c594-47d1-96a4-dbe4f3d0db64'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:16.288194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-fdaca87d-c594-47d1-96a4-dbe4f3d0db64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef2b3b18b54240f19687b89bbe1226b9', 'request_id': 'req-fdaca87d-c594-47d1-96a4-dbe4f3d0db64'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:16.351317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-fdaca87d-c594-47d1-96a4-dbe4f3d0db64 admin admin] The token cache is being invalidate because project ef2b3b18b54240f19687b89bbe1226b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:45:16.351622 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-fdaca87d-c594-47d1-96a4-dbe4f3d0db64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:16.352415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-fdaca87d-c594-47d1-96a4-dbe4f3d0db64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:16.365762 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 858/1696] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:45:15 2022] DELETE /identity/v3/projects/ef2b3b18b54240f19687b89bbe1226b9 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:16.367054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaca87d-c594-47d1-96a4-dbe4f3d0db64 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:16.372554 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-fdaca87d-c594-47d1-96a4-dbe4f3d0db64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:16.483583 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-5810226d-17b1-4982-88e1-d0a694ee3195 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9ce0fe01f5b46b29b9c5a28ca31ac70'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:16.502586 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-5810226d-17b1-4982-88e1-d0a694ee3195 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9ce0fe01f5b46b29b9c5a28ca31ac70'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:16.503482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-5810226d-17b1-4982-88e1-d0a694ee3195 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9ce0fe01f5b46b29b9c5a28ca31ac70'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:16.535793 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-5810226d-17b1-4982-88e1-d0a694ee3195 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9ce0fe01f5b46b29b9c5a28ca31ac70'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:16.541336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaca87d-c594-47d1-96a4-dbe4f3d0db64 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:16.568240 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a639d-876f-41d4-ab5e-2bdd0186ca4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:16.569177 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a639d-876f-41d4-ab5e-2bdd0186ca4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:16.569457 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a639d-876f-41d4-ab5e-2bdd0186ca4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:16.569728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a639d-876f-41d4-ab5e-2bdd0186ca4b admin admin] PATH_INFO: `/v3/projects/5212908648524c769045482fe4d40c69/users/ca102d4720334b548b089adb37f0cf91/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:16.572506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a639d-876f-41d4-ab5e-2bdd0186ca4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5212908648524c769045482fe4d40c69, user_id=ca102d4720334b548b089adb37f0cf91, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:16.575439 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a639d-876f-41d4-ab5e-2bdd0186ca4b admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:16.612843 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 859/1697] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:16 2022] PUT /identity/v3/projects/5212908648524c769045482fe4d40c69/users/ca102d4720334b548b089adb37f0cf91/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:16.614709 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeaaeb8-604e-45c2-8507-2a64f2fcd80e None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:16.615159 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeaaeb8-604e-45c2-8507-2a64f2fcd80e None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:16.615584 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeaaeb8-604e-45c2-8507-2a64f2fcd80e None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:16.617533 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 860/1698] 10.222.0.15 () {54 vars in 921 bytes} [Mon Jun 27 18:45:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:45:16.619332 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd6fda-5382-4e3a-b6a9-da760888c867 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:16.619781 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd6fda-5382-4e3a-b6a9-da760888c867 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:16.620217 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd6fda-5382-4e3a-b6a9-da760888c867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:16.622609 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-5810226d-17b1-4982-88e1-d0a694ee3195 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9ce0fe01f5b46b29b9c5a28ca31ac70'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:16.631340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 839/1699] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:45:15 2022] DELETE /identity/v3/users/e9ce0fe01f5b46b29b9c5a28ca31ac70 => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:16.633218 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adb29d-afff-4649-b7e0-244125af80b6 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:16.633218 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adb29d-afff-4649-b7e0-244125af80b6 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:16.634164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adb29d-afff-4649-b7e0-244125af80b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:16.650271 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-d6cd6fda-5382-4e3a-b6a9-da760888c867 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:16.655238 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-d6cd6fda-5382-4e3a-b6a9-da760888c867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:16.670937 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-45adb29d-afff-4649-b7e0-244125af80b6 None None] MFA Rules not processed for user `ca102d4720334b548b089adb37f0cf91`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:16.673202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-45adb29d-afff-4649-b7e0-244125af80b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:16.775522 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 840/1700] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:16 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:16.776701 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3275e6a-79a0-44e8-8e89-e9ff474e6b03 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:16.777072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3275e6a-79a0-44e8-8e89-e9ff474e6b03 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:16.777244 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3275e6a-79a0-44e8-8e89-e9ff474e6b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:16.801471 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 861/1701] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:16.803372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b80b46-18e1-452a-9cd4-0688317e4a11 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:16.803623 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b80b46-18e1-452a-9cd4-0688317e4a11 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:16.804294 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b80b46-18e1-452a-9cd4-0688317e4a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:16.813798 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:16.826624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-f3275e6a-79a0-44e8-8e89-e9ff474e6b03 None None] MFA Rules not processed for user `3cd6f0a1286243e4add692db01028205`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:16.828352 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f3275e6a-79a0-44e8-8e89-e9ff474e6b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:16.830274 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-52b80b46-18e1-452a-9cd4-0688317e4a11 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:16.831843 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-52b80b46-18e1-452a-9cd4-0688317e4a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:16.843107 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 862/1702] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:16.843981 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b80b46-18e1-452a-9cd4-0688317e4a11 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:16.845683 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-52b80b46-18e1-452a-9cd4-0688317e4a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:17.012737 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 841/1703] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Jun 27 18:45:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:17.018162 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3f155-3386-40ff-9117-3aa52fc4c18d None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:17.018701 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3f155-3386-40ff-9117-3aa52fc4c18d None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:17.019202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3f155-3386-40ff-9117-3aa52fc4c18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:17.084192 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:17.094039 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b80b46-18e1-452a-9cd4-0688317e4a11 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:17.106153 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-f6c3f155-3386-40ff-9117-3aa52fc4c18d None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:17.107426 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f6c3f155-3386-40ff-9117-3aa52fc4c18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:17.109824 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec5037-e49d-44dd-9219-a12344fcecf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:17.110704 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec5037-e49d-44dd-9219-a12344fcecf9 service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:17.110994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec5037-e49d-44dd-9219-a12344fcecf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:17.111276 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec5037-e49d-44dd-9219-a12344fcecf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:17.129829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec5037-e49d-44dd-9219-a12344fcecf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:17.136857 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 842/1704] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:17.136857 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3f155-3386-40ff-9117-3aa52fc4c18d None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:17.137025 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec5037-e49d-44dd-9219-a12344fcecf9 service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:17.141755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f6c3f155-3386-40ff-9117-3aa52fc4c18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:17.157961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 863/1705] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:17.160928 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64045b0f-1707-4261-8da7-7c4150d87c33 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:17.161224 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64045b0f-1707-4261-8da7-7c4150d87c33 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:17.161509 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64045b0f-1707-4261-8da7-7c4150d87c33 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:17.167111 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 864/1706] 10.222.0.15 () {54 vars in 960 bytes} [Mon Jun 27 18:45:17 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:45:17.168194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64045b0f-1707-4261-8da7-7c4150d87c33 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:17.205748 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64045b0f-1707-4261-8da7-7c4150d87c33 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:17.221039 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad1f76-3d71-4682-b346-8e02620d01e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:17.223115 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad1f76-3d71-4682-b346-8e02620d01e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:17.223115 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad1f76-3d71-4682-b346-8e02620d01e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:17.223115 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad1f76-3d71-4682-b346-8e02620d01e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:17.223393 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad1f76-3d71-4682-b346-8e02620d01e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:45:17.223662 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad1f76-3d71-4682-b346-8e02620d01e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:17.229645 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad1f76-3d71-4682-b346-8e02620d01e6 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:17.258047 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 865/1707] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:45:17.258047 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16300f48-a564-49db-afe3-664000414d94 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:17.258047 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16300f48-a564-49db-afe3-664000414d94 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:17.258047 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16300f48-a564-49db-afe3-664000414d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:17.303648 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-16300f48-a564-49db-afe3-664000414d94 None None] MFA Rules not processed for user `3cd6f0a1286243e4add692db01028205`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:17.304709 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-16300f48-a564-49db-afe3-664000414d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:17.311837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3f155-3386-40ff-9117-3aa52fc4c18d None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:17.318554 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce49df-da48-495a-a565-9aa3de88477b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:17.320259 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce49df-da48-495a-a565-9aa3de88477b service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:17.321649 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce49df-da48-495a-a565-9aa3de88477b service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:17.321992 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce49df-da48-495a-a565-9aa3de88477b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:17.336248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 866/1708] 10.222.0.15 () {58 vars in 1057 bytes} [Mon Jun 27 18:45:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:17.346159 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16300f48-a564-49db-afe3-664000414d94 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:17.351195 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ce49df-da48-495a-a565-9aa3de88477b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:17.353763 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ce49df-da48-495a-a565-9aa3de88477b service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:17.359776 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16300f48-a564-49db-afe3-664000414d94 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:17.376128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2ced1-cbb8-47c6-9d3a-414c0069bea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:17.376128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2ced1-cbb8-47c6-9d3a-414c0069bea7 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:17.376360 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2ced1-cbb8-47c6-9d3a-414c0069bea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:17.376576 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2ced1-cbb8-47c6-9d3a-414c0069bea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:17.377293 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2ced1-cbb8-47c6-9d3a-414c0069bea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:17.379291 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2ced1-cbb8-47c6-9d3a-414c0069bea7 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:17.380990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 843/1709] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:17.382035 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce49df-da48-495a-a565-9aa3de88477b service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:17.409769 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce49df-da48-495a-a565-9aa3de88477b service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:17.425254 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:17.435384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d932a-f203-48f2-9b18-180e0c035221 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cd6f0a1286243e4add692db01028205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:17.436418 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d932a-f203-48f2-9b18-180e0c035221 service placement] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:17.436711 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d932a-f203-48f2-9b18-180e0c035221 service placement] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:17.436984 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d932a-f203-48f2-9b18-180e0c035221 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:17.485722 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d932a-f203-48f2-9b18-180e0c035221 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:17.493645 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909d932a-f203-48f2-9b18-180e0c035221 service placement] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:17.518645 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 867/1710] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:17 2022] POST /identity/v3/projects => generated 393 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:17.519901 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2ced1-cbb8-47c6-9d3a-414c0069bea7 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:17.537566 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 844/1711] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:17.538626 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d932a-f203-48f2-9b18-180e0c035221 service placement] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:17.564081 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d932a-f203-48f2-9b18-180e0c035221 service placement] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:17.693081 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2ced1-cbb8-47c6-9d3a-414c0069bea7 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:17.708718 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800f2a3-c85d-4d1c-809c-b98dd5395b8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:17.709391 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800f2a3-c85d-4d1c-809c-b98dd5395b8b service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:17.709677 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800f2a3-c85d-4d1c-809c-b98dd5395b8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:17.710987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800f2a3-c85d-4d1c-809c-b98dd5395b8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:17.716275 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-d800f2a3-c85d-4d1c-809c-b98dd5395b8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:17.720539 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae46d8-8313-4f77-b6ad-559c3b37b7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:17.721296 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae46d8-8313-4f77-b6ad-559c3b37b7d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:17.721582 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae46d8-8313-4f77-b6ad-559c3b37b7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:17.721861 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae46d8-8313-4f77-b6ad-559c3b37b7d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:17.723680 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae46d8-8313-4f77-b6ad-559c3b37b7d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:17.729404 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae46d8-8313-4f77-b6ad-559c3b37b7d0 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:17.831665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800f2a3-c85d-4d1c-809c-b98dd5395b8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:17.833403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800f2a3-c85d-4d1c-809c-b98dd5395b8b service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:17.847284 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 868/1712] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:17.867459 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 845/1713] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:17 2022] POST /identity/v3/users => generated 297 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:17.870229 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800f2a3-c85d-4d1c-809c-b98dd5395b8b service cinder] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:17.904048 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800f2a3-c85d-4d1c-809c-b98dd5395b8b service cinder] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:17.928018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60d00a-2dc2-4183-98c6-6b7653fa75f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:17.928018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60d00a-2dc2-4183-98c6-6b7653fa75f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:17.928018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60d00a-2dc2-4183-98c6-6b7653fa75f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:17.928018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60d00a-2dc2-4183-98c6-6b7653fa75f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:17.928399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60d00a-2dc2-4183-98c6-6b7653fa75f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:17.929470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60d00a-2dc2-4183-98c6-6b7653fa75f4 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:17.961408 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 869/1714] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:17 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:17.965234 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae46d8-8313-4f77-b6ad-559c3b37b7d0 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:17.989079 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae46d8-8313-4f77-b6ad-559c3b37b7d0 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:18.025890 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f7690-927d-4d97-8016-f0c1eb31dfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:18.029923 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f7690-927d-4d97-8016-f0c1eb31dfb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:18.030244 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f7690-927d-4d97-8016-f0c1eb31dfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:18.031265 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f7690-927d-4d97-8016-f0c1eb31dfb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:18.033263 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f7690-927d-4d97-8016-f0c1eb31dfb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:18.034948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f7690-927d-4d97-8016-f0c1eb31dfb9 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:18.082629 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 846/1715] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:17 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:45:18.087929 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60d00a-2dc2-4183-98c6-6b7653fa75f4 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:18.104739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60d00a-2dc2-4183-98c6-6b7653fa75f4 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:18.130040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a0f5e-1371-4e6b-8fd5-42e611c4bae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:18.131292 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a0f5e-1371-4e6b-8fd5-42e611c4bae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:18.131805 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a0f5e-1371-4e6b-8fd5-42e611c4bae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:18.132241 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a0f5e-1371-4e6b-8fd5-42e611c4bae0 admin admin] PATH_INFO: `/v3/projects/6fd170766ea74cd88a87acfbd1687a05/users/d42d132ffa6d4744b20415827926dcbf/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:18.173269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a0f5e-1371-4e6b-8fd5-42e611c4bae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fd170766ea74cd88a87acfbd1687a05, user_id=d42d132ffa6d4744b20415827926dcbf, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:18.176045 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a0f5e-1371-4e6b-8fd5-42e611c4bae0 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:18.276722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 870/1716] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:18 2022] PUT /identity/v3/projects/6fd170766ea74cd88a87acfbd1687a05/users/d42d132ffa6d4744b20415827926dcbf/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:18.279080 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58363f5-c6a4-4dc3-b9fc-254b9a280903 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:18.279507 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58363f5-c6a4-4dc3-b9fc-254b9a280903 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:18.280005 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58363f5-c6a4-4dc3-b9fc-254b9a280903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:18.347768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:18.466923 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-f58363f5-c6a4-4dc3-b9fc-254b9a280903 None None] MFA Rules not processed for user `d42d132ffa6d4744b20415827926dcbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:18.472947 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f58363f5-c6a4-4dc3-b9fc-254b9a280903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:18.627956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:18.652553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 847/1717] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:18.656811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a0f5e-1371-4e6b-8fd5-42e611c4bae0 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:18.832134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a0f5e-1371-4e6b-8fd5-42e611c4bae0 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:18.855497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b325a1-1aee-43d8-ba7a-ae5cd530e161 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:18.855792 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b325a1-1aee-43d8-ba7a-ae5cd530e161 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:18.856044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b325a1-1aee-43d8-ba7a-ae5cd530e161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:18.859275 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50fb5f-2c25-4d55-93cf-db59e6f4638b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:18.859947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50fb5f-2c25-4d55-93cf-db59e6f4638b admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:18.860230 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50fb5f-2c25-4d55-93cf-db59e6f4638b admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:18.862259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50fb5f-2c25-4d55-93cf-db59e6f4638b admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:18.874159 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50fb5f-2c25-4d55-93cf-db59e6f4638b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:18.884428 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50fb5f-2c25-4d55-93cf-db59e6f4638b admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:18.927831 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-18b325a1-1aee-43d8-ba7a-ae5cd530e161 None None] MFA Rules not processed for user `daef56aadd1c4edfa6eeadea63050d37`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:18.931369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-18b325a1-1aee-43d8-ba7a-ae5cd530e161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:19.049401 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 848/1718] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:19.051191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b325a1-1aee-43d8-ba7a-ae5cd530e161 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:19.052757 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-18b325a1-1aee-43d8-ba7a-ae5cd530e161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:19.108385 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b325a1-1aee-43d8-ba7a-ae5cd530e161 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:19.116392 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 871/1719] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:18 2022] POST /identity/v3/projects => generated 393 bytes in 460 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:19.119015 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7e911-3bd9-428c-b1be-774de3ba2524 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:19.119015 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7e911-3bd9-428c-b1be-774de3ba2524 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:19.119015 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7e911-3bd9-428c-b1be-774de3ba2524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:19.171201 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-99d7e911-3bd9-428c-b1be-774de3ba2524 None None] MFA Rules not processed for user `daef56aadd1c4edfa6eeadea63050d37`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:19.172207 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-99d7e911-3bd9-428c-b1be-774de3ba2524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:19.238106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd5d31-7ff1-419e-96d5-7a1375cceaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:19.239532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd5d31-7ff1-419e-96d5-7a1375cceaa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:19.239899 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd5d31-7ff1-419e-96d5-7a1375cceaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:19.240251 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd5d31-7ff1-419e-96d5-7a1375cceaa0 admin admin] PATH_INFO: `/v3/projects/be721490bd6c4084a7d13c39a92e0c05` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:19.241949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd5d31-7ff1-419e-96d5-7a1375cceaa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be721490bd6c4084a7d13c39a92e0c05)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:19.244403 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: WARNING py.warnings [None req-8fcd5d31-7ff1-419e-96d5-7a1375cceaa0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 18:45:19.244403 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: warnings.warn(msg) Jun 27 18:45:19.244403 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: Jun 27 18:45:19.246934 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd5d31-7ff1-419e-96d5-7a1375cceaa0 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:19.356849 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 872/1720] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:19.358864 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7e911-3bd9-428c-b1be-774de3ba2524 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:19.378902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7e911-3bd9-428c-b1be-774de3ba2524 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:19.385278 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5081fe7-a26c-4fb6-8d98-3b459cd42d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:19.386358 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5081fe7-a26c-4fb6-8d98-3b459cd42d39 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:19.386753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5081fe7-a26c-4fb6-8d98-3b459cd42d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:19.387242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5081fe7-a26c-4fb6-8d98-3b459cd42d39 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:19.388025 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5081fe7-a26c-4fb6-8d98-3b459cd42d39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:19.391369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5081fe7-a26c-4fb6-8d98-3b459cd42d39 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:19.548006 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 873/1721] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:19 2022] POST /identity/v3/users => generated 303 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:19.548006 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182e816-a087-41ef-96de-6a993f9371d9 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:19.548006 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182e816-a087-41ef-96de-6a993f9371d9 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:19.548322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182e816-a087-41ef-96de-6a993f9371d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:19.643713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-8fcd5d31-7ff1-419e-96d5-7a1375cceaa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be721490bd6c4084a7d13c39a92e0c05', 'request_id': 'req-8fcd5d31-7ff1-419e-96d5-7a1375cceaa0'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:19.650812 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-7182e816-a087-41ef-96de-6a993f9371d9 None None] MFA Rules not processed for user `f6d74ace9f674f0da40cb35f39d559c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:19.652061 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-7182e816-a087-41ef-96de-6a993f9371d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:19.771139 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-8fcd5d31-7ff1-419e-96d5-7a1375cceaa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be721490bd6c4084a7d13c39a92e0c05', 'request_id': 'req-8fcd5d31-7ff1-419e-96d5-7a1375cceaa0'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:19.795874 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 874/1722] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:19.802494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182e816-a087-41ef-96de-6a993f9371d9 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:19.808005 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-7182e816-a087-41ef-96de-6a993f9371d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:19.825519 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-8fcd5d31-7ff1-419e-96d5-7a1375cceaa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be721490bd6c4084a7d13c39a92e0c05', 'request_id': 'req-8fcd5d31-7ff1-419e-96d5-7a1375cceaa0'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:19.850310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-8fcd5d31-7ff1-419e-96d5-7a1375cceaa0 admin admin] The token cache is being invalidate because project be721490bd6c4084a7d13c39a92e0c05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:45:19.850310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-8fcd5d31-7ff1-419e-96d5-7a1375cceaa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:19.853389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-8fcd5d31-7ff1-419e-96d5-7a1375cceaa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:19.858607 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 849/1723] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:45:19 2022] DELETE /identity/v3/projects/be721490bd6c4084a7d13c39a92e0c05 => generated 0 bytes in 806 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:19.865664 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd5d31-7ff1-419e-96d5-7a1375cceaa0 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:19.871136 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-8fcd5d31-7ff1-419e-96d5-7a1375cceaa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:19.995619 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182e816-a087-41ef-96de-6a993f9371d9 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:19.997264 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd5d31-7ff1-419e-96d5-7a1375cceaa0 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:20.004186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedca6fe-fe33-4e46-a4bd-337e408b1ea3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:20.005061 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedca6fe-fe33-4e46-a4bd-337e408b1ea3 service nova] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:20.005418 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedca6fe-fe33-4e46-a4bd-337e408b1ea3 service nova] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:20.005790 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedca6fe-fe33-4e46-a4bd-337e408b1ea3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:20.008142 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-eedca6fe-fe33-4e46-a4bd-337e408b1ea3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:20.017039 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59df78-3e17-4feb-8d4b-4d8a9813e9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:20.017039 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59df78-3e17-4feb-8d4b-4d8a9813e9ef admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:20.017039 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59df78-3e17-4feb-8d4b-4d8a9813e9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:20.017039 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59df78-3e17-4feb-8d4b-4d8a9813e9ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:20.017730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb59df78-3e17-4feb-8d4b-4d8a9813e9ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:20.017730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb59df78-3e17-4feb-8d4b-4d8a9813e9ef admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:20.045898 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 850/1724] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:20.045898 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59df78-3e17-4feb-8d4b-4d8a9813e9ef admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:20.048420 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-cb59df78-3e17-4feb-8d4b-4d8a9813e9ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:20.072044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59df78-3e17-4feb-8d4b-4d8a9813e9ef admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:20.088089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19115b75-ff61-4248-a1a9-63ef2934b0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:20.089287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19115b75-ff61-4248-a1a9-63ef2934b0f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:20.089730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19115b75-ff61-4248-a1a9-63ef2934b0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:20.090161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19115b75-ff61-4248-a1a9-63ef2934b0f1 admin admin] PATH_INFO: `/v3/users/6447e08e8ac342c0a51a74b5c70262a8` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:20.094149 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19115b75-ff61-4248-a1a9-63ef2934b0f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6447e08e8ac342c0a51a74b5c70262a8)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:20.097229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19115b75-ff61-4248-a1a9-63ef2934b0f1 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:20.129393 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:20.180557 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedca6fe-fe33-4e46-a4bd-337e408b1ea3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:20.183501 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedca6fe-fe33-4e46-a4bd-337e408b1ea3 service nova] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:20.211193 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 875/1725] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:20.213213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedca6fe-fe33-4e46-a4bd-337e408b1ea3 service nova] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:20.218014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-eedca6fe-fe33-4e46-a4bd-337e408b1ea3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:20.277086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedca6fe-fe33-4e46-a4bd-337e408b1ea3 service nova] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:20.303878 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e7e3c-2d55-4663-a395-858cea48479b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:20.304969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e7e3c-2d55-4663-a395-858cea48479b service nova] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:20.305548 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e7e3c-2d55-4663-a395-858cea48479b service nova] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:20.306173 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e7e3c-2d55-4663-a395-858cea48479b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:20.311056 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-9f3e7e3c-2d55-4663-a395-858cea48479b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:20.365838 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-19115b75-ff61-4248-a1a9-63ef2934b0f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6447e08e8ac342c0a51a74b5c70262a8'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:20.367104 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e7e3c-2d55-4663-a395-858cea48479b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:20.372296 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e7e3c-2d55-4663-a395-858cea48479b service nova] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:20.394258 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-19115b75-ff61-4248-a1a9-63ef2934b0f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6447e08e8ac342c0a51a74b5c70262a8'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:20.401259 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-19115b75-ff61-4248-a1a9-63ef2934b0f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6447e08e8ac342c0a51a74b5c70262a8'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:20.432504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 876/1726] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:20.438909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fc11a-e9b7-476a-82fd-797b50fbc135 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:20.444087 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fc11a-e9b7-476a-82fd-797b50fbc135 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:20.445660 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fc11a-e9b7-476a-82fd-797b50fbc135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:20.473622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-19115b75-ff61-4248-a1a9-63ef2934b0f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6447e08e8ac342c0a51a74b5c70262a8'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:20.520437 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-ec6fc11a-e9b7-476a-82fd-797b50fbc135 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:20.523285 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ec6fc11a-e9b7-476a-82fd-797b50fbc135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:20.550318 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 877/1727] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:20.560867 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fc11a-e9b7-476a-82fd-797b50fbc135 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:20.593633 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-19115b75-ff61-4248-a1a9-63ef2934b0f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6447e08e8ac342c0a51a74b5c70262a8'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:20.596746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fc11a-e9b7-476a-82fd-797b50fbc135 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:20.603081 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 851/1728] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:45:20 2022] DELETE /identity/v3/users/6447e08e8ac342c0a51a74b5c70262a8 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:20.603081 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2060de9-d69f-42fb-9389-3d9a789db660 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:20.603081 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2060de9-d69f-42fb-9389-3d9a789db660 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:20.603703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2060de9-d69f-42fb-9389-3d9a789db660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:20.639687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-f2060de9-d69f-42fb-9389-3d9a789db660 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:20.643462 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f2060de9-d69f-42fb-9389-3d9a789db660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:20.748368 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79082c1-0be4-4922-b1fb-d57b883399cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:20.749420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79082c1-0be4-4922-b1fb-d57b883399cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:20.749773 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79082c1-0be4-4922-b1fb-d57b883399cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:20.750116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79082c1-0be4-4922-b1fb-d57b883399cc admin admin] PATH_INFO: `/v3/projects/7981a4af2112498dae3233268827f781/users/f341e7a88d144b52aabecc3887c6ba0b/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:20.792777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79082c1-0be4-4922-b1fb-d57b883399cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7981a4af2112498dae3233268827f781, user_id=f341e7a88d144b52aabecc3887c6ba0b, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:20.795886 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79082c1-0be4-4922-b1fb-d57b883399cc admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:20.841106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 852/1729] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:20.846994 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2060de9-d69f-42fb-9389-3d9a789db660 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:20.869737 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2060de9-d69f-42fb-9389-3d9a789db660 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:20.886617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2727851-1b9c-48c1-85f4-5a7d8fca9659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:20.887554 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2727851-1b9c-48c1-85f4-5a7d8fca9659 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:20.888006 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2727851-1b9c-48c1-85f4-5a7d8fca9659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:20.888426 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2727851-1b9c-48c1-85f4-5a7d8fca9659 admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:20.899945 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2727851-1b9c-48c1-85f4-5a7d8fca9659 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:45:20.901520 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2727851-1b9c-48c1-85f4-5a7d8fca9659 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:20.907292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2727851-1b9c-48c1-85f4-5a7d8fca9659 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:20.929179 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 878/1730] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:20 2022] PUT /identity/v3/projects/7981a4af2112498dae3233268827f781/users/f341e7a88d144b52aabecc3887c6ba0b/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:20.933212 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79082c1-0be4-4922-b1fb-d57b883399cc admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:20.938516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-e79082c1-0be4-4922-b1fb-d57b883399cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:20.969684 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 853/1731] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:45:20.974009 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2727851-1b9c-48c1-85f4-5a7d8fca9659 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:21.001968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2727851-1b9c-48c1-85f4-5a7d8fca9659 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:21.042852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:21.187198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f4890-a9e2-4a1e-bc6c-1c34e65d869e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:21.187891 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f4890-a9e2-4a1e-bc6c-1c34e65d869e admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:21.188772 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f4890-a9e2-4a1e-bc6c-1c34e65d869e admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:21.189218 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f4890-a9e2-4a1e-bc6c-1c34e65d869e admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:21.190120 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f4890-a9e2-4a1e-bc6c-1c34e65d869e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:21.194663 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f4890-a9e2-4a1e-bc6c-1c34e65d869e admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:21.205336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79082c1-0be4-4922-b1fb-d57b883399cc admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:21.218197 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783897a-9c0c-4f6e-bbe6-fe24e15a9ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:21.220714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783897a-9c0c-4f6e-bbe6-fe24e15a9ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:21.222073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783897a-9c0c-4f6e-bbe6-fe24e15a9ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:21.222584 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783897a-9c0c-4f6e-bbe6-fe24e15a9ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:21.225160 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783897a-9c0c-4f6e-bbe6-fe24e15a9ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:21.230656 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783897a-9c0c-4f6e-bbe6-fe24e15a9ef7 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:21.257431 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 879/1732] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:45:21.261517 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783897a-9c0c-4f6e-bbe6-fe24e15a9ef7 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:21.284763 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783897a-9c0c-4f6e-bbe6-fe24e15a9ef7 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:21.293411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ff37e-b7fe-4684-bc2c-e182b2b61397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:21.294597 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ff37e-b7fe-4684-bc2c-e182b2b61397 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:21.295550 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ff37e-b7fe-4684-bc2c-e182b2b61397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:21.295936 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ff37e-b7fe-4684-bc2c-e182b2b61397 admin admin] PATH_INFO: `/v3/projects/7981a4af2112498dae3233268827f781/users/f341e7a88d144b52aabecc3887c6ba0b/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:21.298117 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ff37e-b7fe-4684-bc2c-e182b2b61397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7981a4af2112498dae3233268827f781, user_id=f341e7a88d144b52aabecc3887c6ba0b, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:21.300973 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ff37e-b7fe-4684-bc2c-e182b2b61397 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:21.334106 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 880/1733] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:21 2022] PUT /identity/v3/projects/7981a4af2112498dae3233268827f781/users/f341e7a88d144b52aabecc3887c6ba0b/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:21.344782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2bed5-56f9-4d25-8e0e-b65dd977e497 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:21.345306 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2bed5-56f9-4d25-8e0e-b65dd977e497 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:21.346129 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2bed5-56f9-4d25-8e0e-b65dd977e497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:21.403636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 854/1734] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:20 2022] POST /identity/v3/projects => generated 365 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:45:21.408391 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f4890-a9e2-4a1e-bc6c-1c34e65d869e admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:21.449696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-6fa2bed5-56f9-4d25-8e0e-b65dd977e497 None None] MFA Rules not processed for user `f341e7a88d144b52aabecc3887c6ba0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:21.450967 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-6fa2bed5-56f9-4d25-8e0e-b65dd977e497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:21.584080 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f4890-a9e2-4a1e-bc6c-1c34e65d869e admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:21.597862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb6b26-d5ff-4b05-a0d3-d331fccd1a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:21.598623 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb6b26-d5ff-4b05-a0d3-d331fccd1a90 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:21.598904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb6b26-d5ff-4b05-a0d3-d331fccd1a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:21.599179 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb6b26-d5ff-4b05-a0d3-d331fccd1a90 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:21.599937 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb6b26-d5ff-4b05-a0d3-d331fccd1a90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:21.603936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb6b26-d5ff-4b05-a0d3-d331fccd1a90 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:21.733016 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 855/1735] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:21 2022] POST /identity/v3/users => generated 283 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:21.741703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc43a8-c528-4569-881c-0f929348ee07 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:21.742144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc43a8-c528-4569-881c-0f929348ee07 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:21.748073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 881/1736] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:21.748222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2bed5-56f9-4d25-8e0e-b65dd977e497 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:21.750703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc43a8-c528-4569-881c-0f929348ee07 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:21.755527 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 856/1737] 10.222.0.15 () {54 vars in 912 bytes} [Mon Jun 27 18:45:21 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:45:21.763094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d274d-6812-4f2c-a7e8-9ff470a6a059 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:21.764705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d274d-6812-4f2c-a7e8-9ff470a6a059 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:21.764705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d274d-6812-4f2c-a7e8-9ff470a6a059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:21.770703 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2bed5-56f9-4d25-8e0e-b65dd977e497 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:21.788202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43219f-c765-4945-bbdc-bf5084205fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:21.788913 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43219f-c765-4945-bbdc-bf5084205fec admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:21.789199 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43219f-c765-4945-bbdc-bf5084205fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:21.790103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43219f-c765-4945-bbdc-bf5084205fec admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:21.790970 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43219f-c765-4945-bbdc-bf5084205fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:21.792745 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43219f-c765-4945-bbdc-bf5084205fec admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:21.817687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 882/1738] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:21.818703 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43219f-c765-4945-bbdc-bf5084205fec admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:21.841378 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ba43219f-c765-4945-bbdc-bf5084205fec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:21.842702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-b53d274d-6812-4f2c-a7e8-9ff470a6a059 None None] MFA Rules not processed for user `dbe621916dbe4f1a8686097fe2bdf762`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:21.845004 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-b53d274d-6812-4f2c-a7e8-9ff470a6a059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:21.969007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 857/1739] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Jun 27 18:45:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:21.970135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d274d-6812-4f2c-a7e8-9ff470a6a059 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:21.984009 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d274d-6812-4f2c-a7e8-9ff470a6a059 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:21.991601 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43219f-c765-4945-bbdc-bf5084205fec admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:21.992001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595194a-8355-4e02-92a0-dada8fc16065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:21.993017 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595194a-8355-4e02-92a0-dada8fc16065 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:21.993482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595194a-8355-4e02-92a0-dada8fc16065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:21.993976 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595194a-8355-4e02-92a0-dada8fc16065 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:21.994907 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595194a-8355-4e02-92a0-dada8fc16065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:21.997021 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595194a-8355-4e02-92a0-dada8fc16065 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:22.002577 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e878662-0f3a-496a-b4fa-8c6c0820229a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:22.002725 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e878662-0f3a-496a-b4fa-8c6c0820229a service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:22.002971 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e878662-0f3a-496a-b4fa-8c6c0820229a service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:22.003252 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e878662-0f3a-496a-b4fa-8c6c0820229a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:22.019590 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e878662-0f3a-496a-b4fa-8c6c0820229a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:22.022092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e878662-0f3a-496a-b4fa-8c6c0820229a service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:22.026355 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 858/1740] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:22.027570 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595194a-8355-4e02-92a0-dada8fc16065 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:22.030706 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-e595194a-8355-4e02-92a0-dada8fc16065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:22.050034 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595194a-8355-4e02-92a0-dada8fc16065 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:22.058492 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 883/1741] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:22.062100 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e878662-0f3a-496a-b4fa-8c6c0820229a service cinder] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:22.063404 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51c2b0-f246-4afd-bda4-c8d62a2c5f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:22.064461 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51c2b0-f246-4afd-bda4-c8d62a2c5f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:22.064902 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51c2b0-f246-4afd-bda4-c8d62a2c5f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:22.065358 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51c2b0-f246-4afd-bda4-c8d62a2c5f4f admin admin] PATH_INFO: `/v3/projects/b1d1dd35ba3440729652656271622d6d` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:22.067069 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51c2b0-f246-4afd-bda4-c8d62a2c5f4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1d1dd35ba3440729652656271622d6d)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:22.068739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-3e878662-0f3a-496a-b4fa-8c6c0820229a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:22.071385 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51c2b0-f246-4afd-bda4-c8d62a2c5f4f admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:22.105485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e878662-0f3a-496a-b4fa-8c6c0820229a service cinder] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:22.115038 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07398a-53d0-423c-b1e6-999a1254e491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:22.116754 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07398a-53d0-423c-b1e6-999a1254e491 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:22.116754 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07398a-53d0-423c-b1e6-999a1254e491 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:22.116754 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07398a-53d0-423c-b1e6-999a1254e491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:22.158924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b07398a-53d0-423c-b1e6-999a1254e491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:22.160514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b07398a-53d0-423c-b1e6-999a1254e491 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:22.176781 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 884/1742] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:22.178099 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07398a-53d0-423c-b1e6-999a1254e491 service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:22.196542 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07398a-53d0-423c-b1e6-999a1254e491 service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:22.209886 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f28ff-a01b-4c6e-a75a-a1d3a7b5b39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:22.210929 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f28ff-a01b-4c6e-a75a-a1d3a7b5b39c admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:22.211218 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f28ff-a01b-4c6e-a75a-a1d3a7b5b39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:22.212021 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f28ff-a01b-4c6e-a75a-a1d3a7b5b39c admin admin] PATH_INFO: `/v3/projects/6509d0e8c91a4d86b6f58a0feb725064/users/120aa1da5ea0447bab7d3d27d808e581/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:22.244813 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-da51c2b0-f246-4afd-bda4-c8d62a2c5f4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1d1dd35ba3440729652656271622d6d', 'request_id': 'req-da51c2b0-f246-4afd-bda4-c8d62a2c5f4f'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:22.249652 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f28ff-a01b-4c6e-a75a-a1d3a7b5b39c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6509d0e8c91a4d86b6f58a0feb725064, user_id=120aa1da5ea0447bab7d3d27d808e581, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:22.252669 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f28ff-a01b-4c6e-a75a-a1d3a7b5b39c admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:22.347533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-da51c2b0-f246-4afd-bda4-c8d62a2c5f4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1d1dd35ba3440729652656271622d6d', 'request_id': 'req-da51c2b0-f246-4afd-bda4-c8d62a2c5f4f'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:22.356892 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-da51c2b0-f246-4afd-bda4-c8d62a2c5f4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1d1dd35ba3440729652656271622d6d', 'request_id': 'req-da51c2b0-f246-4afd-bda4-c8d62a2c5f4f'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:22.375625 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-da51c2b0-f246-4afd-bda4-c8d62a2c5f4f admin admin] The token cache is being invalidate because project b1d1dd35ba3440729652656271622d6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:45:22.376010 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-da51c2b0-f246-4afd-bda4-c8d62a2c5f4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:22.377087 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-da51c2b0-f246-4afd-bda4-c8d62a2c5f4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:22.379634 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 859/1743] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:45:22 2022] DELETE /identity/v3/projects/b1d1dd35ba3440729652656271622d6d => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:22.381093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec11cdd-44f7-486f-908c-e9c433da8ecf None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:22.381453 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec11cdd-44f7-486f-908c-e9c433da8ecf None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:22.381804 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec11cdd-44f7-486f-908c-e9c433da8ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:22.402944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 885/1744] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:22 2022] PUT /identity/v3/projects/6509d0e8c91a4d86b6f58a0feb725064/users/120aa1da5ea0447bab7d3d27d808e581/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:22.403755 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f28ff-a01b-4c6e-a75a-a1d3a7b5b39c admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:22.405683 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-045f28ff-a01b-4c6e-a75a-a1d3a7b5b39c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:22.422929 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-bec11cdd-44f7-486f-908c-e9c433da8ecf None None] MFA Rules not processed for user `2fc6711e48be4d4ab33695e06c7227ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:22.425048 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-bec11cdd-44f7-486f-908c-e9c433da8ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:22.495094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f28ff-a01b-4c6e-a75a-a1d3a7b5b39c admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:22.501399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de53157-25d0-4005-bcc6-ea3cfcc95676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:22.502783 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de53157-25d0-4005-bcc6-ea3cfcc95676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:22.503088 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de53157-25d0-4005-bcc6-ea3cfcc95676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:22.503374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de53157-25d0-4005-bcc6-ea3cfcc95676 admin admin] PATH_INFO: `/v3/users/2df94bc89c40436abf617814a6d78df3` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:22.505252 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de53157-25d0-4005-bcc6-ea3cfcc95676 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2df94bc89c40436abf617814a6d78df3)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:22.509567 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de53157-25d0-4005-bcc6-ea3cfcc95676 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:22.520955 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 860/1745] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:22.520955 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec11cdd-44f7-486f-908c-e9c433da8ecf None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:22.520955 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-bec11cdd-44f7-486f-908c-e9c433da8ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:22.604041 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec11cdd-44f7-486f-908c-e9c433da8ecf None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:22.609190 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083a73e-4bad-4e5d-80f4-186237a52801 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:22.609442 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083a73e-4bad-4e5d-80f4-186237a52801 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:22.609855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083a73e-4bad-4e5d-80f4-186237a52801 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:22.610206 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083a73e-4bad-4e5d-80f4-186237a52801 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:22.613360 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-e083a73e-4bad-4e5d-80f4-186237a52801 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:22.627758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:22.835967 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:22.840013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083a73e-4bad-4e5d-80f4-186237a52801 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:22.842060 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083a73e-4bad-4e5d-80f4-186237a52801 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:22.846278 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-2de53157-25d0-4005-bcc6-ea3cfcc95676 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2df94bc89c40436abf617814a6d78df3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:22.884312 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-2de53157-25d0-4005-bcc6-ea3cfcc95676 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2df94bc89c40436abf617814a6d78df3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:22.885250 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-2de53157-25d0-4005-bcc6-ea3cfcc95676 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2df94bc89c40436abf617814a6d78df3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:22.887765 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-2de53157-25d0-4005-bcc6-ea3cfcc95676 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2df94bc89c40436abf617814a6d78df3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:22.920598 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-2de53157-25d0-4005-bcc6-ea3cfcc95676 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2df94bc89c40436abf617814a6d78df3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:22.927368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 861/1746] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:22.931689 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083a73e-4bad-4e5d-80f4-186237a52801 service cinder] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:22.935764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-e083a73e-4bad-4e5d-80f4-186237a52801 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:23.016951 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 886/1747] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:45:22 2022] DELETE /identity/v3/users/2df94bc89c40436abf617814a6d78df3 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:23.024214 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4854382-3ed1-436d-bb7f-ffb287abbc41 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:23.024616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4854382-3ed1-436d-bb7f-ffb287abbc41 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:23.025438 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4854382-3ed1-436d-bb7f-ffb287abbc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:23.032482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083a73e-4bad-4e5d-80f4-186237a52801 service cinder] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:23.135270 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-c4854382-3ed1-436d-bb7f-ffb287abbc41 None None] MFA Rules not processed for user `120aa1da5ea0447bab7d3d27d808e581`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:23.136908 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c4854382-3ed1-436d-bb7f-ffb287abbc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:23.242494 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d9e3a-5a9d-431a-84d2-12f629d970d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:23.246982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d9e3a-5a9d-431a-84d2-12f629d970d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:23.254564 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d9e3a-5a9d-431a-84d2-12f629d970d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:23.254564 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d9e3a-5a9d-431a-84d2-12f629d970d6 admin admin] PATH_INFO: `/v3/users/7be1b248765c4992bc621ce2ab16c0ac` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:23.254564 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d9e3a-5a9d-431a-84d2-12f629d970d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7be1b248765c4992bc621ce2ab16c0ac)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:23.255739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d9e3a-5a9d-431a-84d2-12f629d970d6 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:23.337147 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 887/1748] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:23 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:23.347014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fe7d7-8280-4de3-96c5-10dff308e7ba None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:23.347491 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fe7d7-8280-4de3-96c5-10dff308e7ba None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:23.352464 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9fe7d7-8280-4de3-96c5-10dff308e7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:23.390801 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-3b9fe7d7-8280-4de3-96c5-10dff308e7ba None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:23.392302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-3b9fe7d7-8280-4de3-96c5-10dff308e7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:23.403790 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 888/1749] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:23.403790 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fe7d7-8280-4de3-96c5-10dff308e7ba None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:23.411337 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-3b9fe7d7-8280-4de3-96c5-10dff308e7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:23.509357 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9fe7d7-8280-4de3-96c5-10dff308e7ba None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:23.516762 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc4542-9706-48f2-88eb-3da8a4cffb8c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cd6f0a1286243e4add692db01028205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:23.518182 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc4542-9706-48f2-88eb-3da8a4cffb8c service placement] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:23.518565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc4542-9706-48f2-88eb-3da8a4cffb8c service placement] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:23.521508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc4542-9706-48f2-88eb-3da8a4cffb8c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:23.521508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-f5dc4542-9706-48f2-88eb-3da8a4cffb8c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:23.568114 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc4542-9706-48f2-88eb-3da8a4cffb8c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:23.568764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-860d9e3a-5a9d-431a-84d2-12f629d970d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7be1b248765c4992bc621ce2ab16c0ac'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:23.580563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc4542-9706-48f2-88eb-3da8a4cffb8c service placement] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:23.590357 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-860d9e3a-5a9d-431a-84d2-12f629d970d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7be1b248765c4992bc621ce2ab16c0ac'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:23.592281 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-860d9e3a-5a9d-431a-84d2-12f629d970d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7be1b248765c4992bc621ce2ab16c0ac'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:23.603657 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 889/1750] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:23.604607 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc4542-9706-48f2-88eb-3da8a4cffb8c service placement] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:23.609669 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-f5dc4542-9706-48f2-88eb-3da8a4cffb8c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:23.625169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-860d9e3a-5a9d-431a-84d2-12f629d970d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7be1b248765c4992bc621ce2ab16c0ac'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:23.714852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc4542-9706-48f2-88eb-3da8a4cffb8c service placement] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:23.720763 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b18d13-36aa-4767-ab5f-73296599d09c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:23.721424 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b18d13-36aa-4767-ab5f-73296599d09c service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:23.721712 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b18d13-36aa-4767-ab5f-73296599d09c service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:23.721995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b18d13-36aa-4767-ab5f-73296599d09c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:23.739738 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-860d9e3a-5a9d-431a-84d2-12f629d970d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7be1b248765c4992bc621ce2ab16c0ac'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:23.746704 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b18d13-36aa-4767-ab5f-73296599d09c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:23.748252 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b18d13-36aa-4767-ab5f-73296599d09c service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:23.758722 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 862/1751] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:45:22 2022] DELETE /identity/v3/users/7be1b248765c4992bc621ce2ab16c0ac => generated 0 bytes in 830 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:23.764908 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96080ab-7f14-4a3e-8b88-0ad48061c0b3 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:23.765303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96080ab-7f14-4a3e-8b88-0ad48061c0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:23.765652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96080ab-7f14-4a3e-8b88-0ad48061c0b3 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:23.767588 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 863/1752] 10.222.0.15 () {54 vars in 912 bytes} [Mon Jun 27 18:45:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:45:23.773113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96080ab-7f14-4a3e-8b88-0ad48061c0b3 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:23.773226 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 890/1753] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:23.775255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd3a36-4748-4bf7-8b13-4c322ec4a062 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:23.775527 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-e96080ab-7f14-4a3e-8b88-0ad48061c0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:23.775638 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd3a36-4748-4bf7-8b13-4c322ec4a062 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:23.775827 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd3a36-4748-4bf7-8b13-4c322ec4a062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:23.824694 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-f3cd3a36-4748-4bf7-8b13-4c322ec4a062 None None] MFA Rules not processed for user `dbe621916dbe4f1a8686097fe2bdf762`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:23.824822 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-f3cd3a36-4748-4bf7-8b13-4c322ec4a062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:23.921303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96080ab-7f14-4a3e-8b88-0ad48061c0b3 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:23.934835 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790794f0-9ce9-4b32-856b-bcfaf2e5a4ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:23.935542 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790794f0-9ce9-4b32-856b-bcfaf2e5a4ea service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:23.935771 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790794f0-9ce9-4b32-856b-bcfaf2e5a4ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:23.936045 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790794f0-9ce9-4b32-856b-bcfaf2e5a4ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:23.940325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 891/1754] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Jun 27 18:45:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:23.953214 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-790794f0-9ce9-4b32-856b-bcfaf2e5a4ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:23.959327 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd3a36-4748-4bf7-8b13-4c322ec4a062 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:23.960762 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-f3cd3a36-4748-4bf7-8b13-4c322ec4a062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:23.981954 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd3a36-4748-4bf7-8b13-4c322ec4a062 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:23.994029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67d76d-e5f7-42f7-8e51-e278fd326a97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:23.994752 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67d76d-e5f7-42f7-8e51-e278fd326a97 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:23.995046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67d76d-e5f7-42f7-8e51-e278fd326a97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:23.995301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67d76d-e5f7-42f7-8e51-e278fd326a97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:24.014665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67d76d-e5f7-42f7-8e51-e278fd326a97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:24.018214 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67d76d-e5f7-42f7-8e51-e278fd326a97 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:24.040840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 892/1755] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:24.088647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790794f0-9ce9-4b32-856b-bcfaf2e5a4ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:24.091824 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790794f0-9ce9-4b32-856b-bcfaf2e5a4ea service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:24.129040 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 864/1756] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:23 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:24.540235 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67d76d-e5f7-42f7-8e51-e278fd326a97 service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:24.542556 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:24.544900 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-df67d76d-e5f7-42f7-8e51-e278fd326a97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:24.748748 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67d76d-e5f7-42f7-8e51-e278fd326a97 service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:24.765062 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e42da8-5ac7-4fc3-8937-4a7abe0a6fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:24.766096 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e42da8-5ac7-4fc3-8937-4a7abe0a6fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:24.766404 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e42da8-5ac7-4fc3-8937-4a7abe0a6fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:24.766685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e42da8-5ac7-4fc3-8937-4a7abe0a6fc4 admin admin] PATH_INFO: `/v3/projects/468d2ead22fa413b90ca7f9934a11233` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:24.802811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e42da8-5ac7-4fc3-8937-4a7abe0a6fc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=468d2ead22fa413b90ca7f9934a11233)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:24.808194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e42da8-5ac7-4fc3-8937-4a7abe0a6fc4 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:24.988131 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-27e42da8-5ac7-4fc3-8937-4a7abe0a6fc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '468d2ead22fa413b90ca7f9934a11233', 'request_id': 'req-27e42da8-5ac7-4fc3-8937-4a7abe0a6fc4'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:24.990707 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790794f0-9ce9-4b32-856b-bcfaf2e5a4ea service cinder] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:24.992424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-790794f0-9ce9-4b32-856b-bcfaf2e5a4ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:25.023014 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790794f0-9ce9-4b32-856b-bcfaf2e5a4ea service cinder] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:25.085326 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-27e42da8-5ac7-4fc3-8937-4a7abe0a6fc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '468d2ead22fa413b90ca7f9934a11233', 'request_id': 'req-27e42da8-5ac7-4fc3-8937-4a7abe0a6fc4'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:25.091237 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465717c-052f-4c51-970c-6e8987805498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:25.092109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465717c-052f-4c51-970c-6e8987805498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:25.092607 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465717c-052f-4c51-970c-6e8987805498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:25.093023 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465717c-052f-4c51-970c-6e8987805498 admin admin] PATH_INFO: `/v3/projects/ee377b3bdfe84289b946d3de63caa410` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:25.094788 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465717c-052f-4c51-970c-6e8987805498 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee377b3bdfe84289b946d3de63caa410)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:25.097378 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465717c-052f-4c51-970c-6e8987805498 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:25.108342 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-27e42da8-5ac7-4fc3-8937-4a7abe0a6fc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '468d2ead22fa413b90ca7f9934a11233', 'request_id': 'req-27e42da8-5ac7-4fc3-8937-4a7abe0a6fc4'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:25.158986 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-27e42da8-5ac7-4fc3-8937-4a7abe0a6fc4 admin admin] The token cache is being invalidate because project 468d2ead22fa413b90ca7f9934a11233 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:45:25.160649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-27e42da8-5ac7-4fc3-8937-4a7abe0a6fc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:25.164553 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-27e42da8-5ac7-4fc3-8937-4a7abe0a6fc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:25.169015 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 893/1757] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:45:24 2022] DELETE /identity/v3/projects/468d2ead22fa413b90ca7f9934a11233 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:25.178431 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a80fc-6218-4f27-b3f5-5a83794e3ed4 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:25.178431 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a80fc-6218-4f27-b3f5-5a83794e3ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:25.178431 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a80fc-6218-4f27-b3f5-5a83794e3ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:25.228483 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-981a80fc-6218-4f27-b3f5-5a83794e3ed4 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:25.233226 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-981a80fc-6218-4f27-b3f5-5a83794e3ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:25.285017 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-3465717c-052f-4c51-970c-6e8987805498 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee377b3bdfe84289b946d3de63caa410', 'request_id': 'req-3465717c-052f-4c51-970c-6e8987805498'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:25.286501 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 894/1758] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:25.294935 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ca198-9083-4e2c-ac9d-d5dbbc385403 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:25.295514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ca198-9083-4e2c-ac9d-d5dbbc385403 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:25.296101 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ca198-9083-4e2c-ac9d-d5dbbc385403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:25.350765 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-248ca198-9083-4e2c-ac9d-d5dbbc385403 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:25.350765 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-248ca198-9083-4e2c-ac9d-d5dbbc385403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:25.393028 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-3465717c-052f-4c51-970c-6e8987805498 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee377b3bdfe84289b946d3de63caa410', 'request_id': 'req-3465717c-052f-4c51-970c-6e8987805498'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:25.401720 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-3465717c-052f-4c51-970c-6e8987805498 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee377b3bdfe84289b946d3de63caa410', 'request_id': 'req-3465717c-052f-4c51-970c-6e8987805498'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:25.430867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-3465717c-052f-4c51-970c-6e8987805498 admin admin] The token cache is being invalidate because project ee377b3bdfe84289b946d3de63caa410 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:45:25.431168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-3465717c-052f-4c51-970c-6e8987805498 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:25.435987 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-3465717c-052f-4c51-970c-6e8987805498 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:25.439171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 865/1759] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:45:24 2022] DELETE /identity/v3/projects/ee377b3bdfe84289b946d3de63caa410 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:25.446242 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465717c-052f-4c51-970c-6e8987805498 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:25.449844 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-3465717c-052f-4c51-970c-6e8987805498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:25.502549 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 895/1760] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:25.503655 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ca198-9083-4e2c-ac9d-d5dbbc385403 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:25.520531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ca198-9083-4e2c-ac9d-d5dbbc385403 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:25.538978 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955211e5-1ef4-469d-aaf5-97c71083390b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:25.538978 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955211e5-1ef4-469d-aaf5-97c71083390b admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:25.538978 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955211e5-1ef4-469d-aaf5-97c71083390b admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:25.538978 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955211e5-1ef4-469d-aaf5-97c71083390b admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:25.539784 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955211e5-1ef4-469d-aaf5-97c71083390b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:45:25.539784 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955211e5-1ef4-469d-aaf5-97c71083390b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:25.540723 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955211e5-1ef4-469d-aaf5-97c71083390b admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:25.565535 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:25.577090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 896/1761] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:45:25.577090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955211e5-1ef4-469d-aaf5-97c71083390b admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:25.602321 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955211e5-1ef4-469d-aaf5-97c71083390b admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:25.615054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718c684-17a3-451a-932e-e9da3cfec0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:25.615729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718c684-17a3-451a-932e-e9da3cfec0b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:25.616012 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718c684-17a3-451a-932e-e9da3cfec0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:25.616287 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718c684-17a3-451a-932e-e9da3cfec0b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:25.616992 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9718c684-17a3-451a-932e-e9da3cfec0b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:25.621119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9718c684-17a3-451a-932e-e9da3cfec0b9 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:25.634213 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465717c-052f-4c51-970c-6e8987805498 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:25.649800 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54de0a8-a74e-4056-9765-a3e0bdf09cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:25.651143 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54de0a8-a74e-4056-9765-a3e0bdf09cd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:25.651562 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54de0a8-a74e-4056-9765-a3e0bdf09cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:25.652005 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54de0a8-a74e-4056-9765-a3e0bdf09cd2 admin admin] PATH_INFO: `/v3/users/987f7e12cd294ddabd634eee97c99b33` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:25.653624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54de0a8-a74e-4056-9765-a3e0bdf09cd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=987f7e12cd294ddabd634eee97c99b33)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:25.656315 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54de0a8-a74e-4056-9765-a3e0bdf09cd2 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:25.779287 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 897/1762] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:25 2022] POST /identity/v3/projects => generated 375 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:25.783337 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718c684-17a3-451a-932e-e9da3cfec0b9 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:25.865632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718c684-17a3-451a-932e-e9da3cfec0b9 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:25.872030 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b858ebf-b1cd-4e01-ad31-2c7c2ca9cd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:25.872803 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b858ebf-b1cd-4e01-ad31-2c7c2ca9cd87 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:25.873445 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b858ebf-b1cd-4e01-ad31-2c7c2ca9cd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:25.873861 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b858ebf-b1cd-4e01-ad31-2c7c2ca9cd87 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:25.874725 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b858ebf-b1cd-4e01-ad31-2c7c2ca9cd87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:25.885001 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b858ebf-b1cd-4e01-ad31-2c7c2ca9cd87 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:25.925482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f54de0a8-a74e-4056-9765-a3e0bdf09cd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '987f7e12cd294ddabd634eee97c99b33'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:25.934647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f54de0a8-a74e-4056-9765-a3e0bdf09cd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '987f7e12cd294ddabd634eee97c99b33'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:25.935245 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f54de0a8-a74e-4056-9765-a3e0bdf09cd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '987f7e12cd294ddabd634eee97c99b33'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:25.987736 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f54de0a8-a74e-4056-9765-a3e0bdf09cd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '987f7e12cd294ddabd634eee97c99b33'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:26.013846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 898/1763] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:25 2022] POST /identity/v3/users => generated 288 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:26.013846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b705b12-d215-467d-84c2-bc2a4897851a None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:26.013846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b705b12-d215-467d-84c2-bc2a4897851a None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:26.013846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b705b12-d215-467d-84c2-bc2a4897851a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:26.042725 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-6b705b12-d215-467d-84c2-bc2a4897851a None None] MFA Rules not processed for user `f6d74ace9f674f0da40cb35f39d559c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:26.043943 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-6b705b12-d215-467d-84c2-bc2a4897851a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:26.084781 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f54de0a8-a74e-4056-9765-a3e0bdf09cd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '987f7e12cd294ddabd634eee97c99b33'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:26.093566 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 866/1764] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:45:25 2022] DELETE /identity/v3/users/987f7e12cd294ddabd634eee97c99b33 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:26.095072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54de0a8-a74e-4056-9765-a3e0bdf09cd2 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:26.097160 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f54de0a8-a74e-4056-9765-a3e0bdf09cd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:26.154449 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 899/1765] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:26.224175 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54de0a8-a74e-4056-9765-a3e0bdf09cd2 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:26.234095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827db7a-5014-4d9f-8afe-32beab72854a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:26.235223 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827db7a-5014-4d9f-8afe-32beab72854a admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:26.237420 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827db7a-5014-4d9f-8afe-32beab72854a admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:26.237420 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827db7a-5014-4d9f-8afe-32beab72854a admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:26.237420 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4827db7a-5014-4d9f-8afe-32beab72854a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:26.239921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4827db7a-5014-4d9f-8afe-32beab72854a admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:26.258663 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 867/1766] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:26.269102 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b705b12-d215-467d-84c2-bc2a4897851a None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:26.286914 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b705b12-d215-467d-84c2-bc2a4897851a None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:26.304623 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62dd6e5-ea75-4891-9e71-0859bef3e2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:26.306956 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62dd6e5-ea75-4891-9e71-0859bef3e2a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:26.307318 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62dd6e5-ea75-4891-9e71-0859bef3e2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:26.307676 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62dd6e5-ea75-4891-9e71-0859bef3e2a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:26.309026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62dd6e5-ea75-4891-9e71-0859bef3e2a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:26.314821 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62dd6e5-ea75-4891-9e71-0859bef3e2a8 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:26.332262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 900/1767] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:26.337057 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827db7a-5014-4d9f-8afe-32beab72854a admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:26.350232 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827db7a-5014-4d9f-8afe-32beab72854a admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:26.351540 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:26.373084 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbaf00-c077-4ab4-97c5-98e40c21ffd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:26.379481 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbaf00-c077-4ab4-97c5-98e40c21ffd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:26.380260 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbaf00-c077-4ab4-97c5-98e40c21ffd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:26.380712 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbaf00-c077-4ab4-97c5-98e40c21ffd9 admin admin] PATH_INFO: `/v3/projects/34fac855cf8146ec8a8553d9c73cc648/users/c78783feb78f42ac92a9766cb4718999/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:26.406044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bbaf00-c077-4ab4-97c5-98e40c21ffd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34fac855cf8146ec8a8553d9c73cc648, user_id=c78783feb78f42ac92a9766cb4718999, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:26.409527 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bbaf00-c077-4ab4-97c5-98e40c21ffd9 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:26.507164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 868/1768] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:26 2022] PUT /identity/v3/projects/34fac855cf8146ec8a8553d9c73cc648/users/c78783feb78f42ac92a9766cb4718999/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:26.514375 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b155250-9967-4d42-9188-1365fb48edb1 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:26.514749 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b155250-9967-4d42-9188-1365fb48edb1 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:26.514983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b155250-9967-4d42-9188-1365fb48edb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:26.544038 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-8b155250-9967-4d42-9188-1365fb48edb1 None None] MFA Rules not processed for user `c78783feb78f42ac92a9766cb4718999`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:26.545295 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-8b155250-9967-4d42-9188-1365fb48edb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:26.630395 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 901/1769] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:26.634640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbaf00-c077-4ab4-97c5-98e40c21ffd9 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:26.764376 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbaf00-c077-4ab4-97c5-98e40c21ffd9 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:26.769836 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a26325-ca04-4bea-a9a4-22e7a826e93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:26.770738 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a26325-ca04-4bea-a9a4-22e7a826e93e admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:26.771198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a26325-ca04-4bea-a9a4-22e7a826e93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:26.771779 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a26325-ca04-4bea-a9a4-22e7a826e93e admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:26.772208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a26325-ca04-4bea-a9a4-22e7a826e93e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:26.775816 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a26325-ca04-4bea-a9a4-22e7a826e93e admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:26.910421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 869/1770] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:26 2022] POST /identity/v3/projects => generated 375 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:26.915108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b155250-9967-4d42-9188-1365fb48edb1 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:26.926333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b155250-9967-4d42-9188-1365fb48edb1 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:26.942233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a26325-ca04-4bea-a9a4-22e7a826e93e admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:26.944244 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-84a26325-ca04-4bea-a9a4-22e7a826e93e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:27.000826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52649e30-b747-4f6a-8ab1-2363daec5bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:27.001799 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52649e30-b747-4f6a-8ab1-2363daec5bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:27.002077 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52649e30-b747-4f6a-8ab1-2363daec5bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:27.002574 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52649e30-b747-4f6a-8ab1-2363daec5bf4 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:27.003534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52649e30-b747-4f6a-8ab1-2363daec5bf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:27.005584 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52649e30-b747-4f6a-8ab1-2363daec5bf4 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:27.022834 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a26325-ca04-4bea-a9a4-22e7a826e93e admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:27.030208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702ef3e-ab5a-4d22-ae4e-eb1ae7b77700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:27.031100 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702ef3e-ab5a-4d22-ae4e-eb1ae7b77700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:27.031384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702ef3e-ab5a-4d22-ae4e-eb1ae7b77700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:27.031745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702ef3e-ab5a-4d22-ae4e-eb1ae7b77700 admin admin] PATH_INFO: `/v3/projects/f239ff88533040a59e227992c5538480` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:27.032841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702ef3e-ab5a-4d22-ae4e-eb1ae7b77700 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f239ff88533040a59e227992c5538480)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:27.035176 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4702ef3e-ab5a-4d22-ae4e-eb1ae7b77700 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:27.124050 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 902/1771] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:26 2022] POST /identity/v3/users => generated 294 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:27.125402 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52649e30-b747-4f6a-8ab1-2363daec5bf4 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:27.136964 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52649e30-b747-4f6a-8ab1-2363daec5bf4 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:27.141835 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c2ccc-81b5-462b-b095-757487cf00bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:27.142632 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c2ccc-81b5-462b-b095-757487cf00bd admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:27.143115 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c2ccc-81b5-462b-b095-757487cf00bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:27.143585 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c2ccc-81b5-462b-b095-757487cf00bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:27.144565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c2ccc-81b5-462b-b095-757487cf00bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:27.146546 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c2ccc-81b5-462b-b095-757487cf00bd admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:27.160123 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 903/1772] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:27.165402 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c2ccc-81b5-462b-b095-757487cf00bd admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:27.184359 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c2ccc-81b5-462b-b095-757487cf00bd admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:27.214349 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-4702ef3e-ab5a-4d22-ae4e-eb1ae7b77700 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f239ff88533040a59e227992c5538480', 'request_id': 'req-4702ef3e-ab5a-4d22-ae4e-eb1ae7b77700'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:27.272941 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd160f-5a75-4852-b16d-2b0d64328490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:27.274124 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd160f-5a75-4852-b16d-2b0d64328490 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:27.274779 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd160f-5a75-4852-b16d-2b0d64328490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:27.275296 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd160f-5a75-4852-b16d-2b0d64328490 admin admin] PATH_INFO: `/v3/projects/3b50437225624a7a916a0256179bf9dc/users/8ec088c3a48f42b1af55d10993d676ee/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:27.301262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd160f-5a75-4852-b16d-2b0d64328490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b50437225624a7a916a0256179bf9dc, user_id=8ec088c3a48f42b1af55d10993d676ee, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:27.305277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd160f-5a75-4852-b16d-2b0d64328490 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:27.306660 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-4702ef3e-ab5a-4d22-ae4e-eb1ae7b77700 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f239ff88533040a59e227992c5538480', 'request_id': 'req-4702ef3e-ab5a-4d22-ae4e-eb1ae7b77700'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:27.329760 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-4702ef3e-ab5a-4d22-ae4e-eb1ae7b77700 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f239ff88533040a59e227992c5538480', 'request_id': 'req-4702ef3e-ab5a-4d22-ae4e-eb1ae7b77700'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:27.341730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-4702ef3e-ab5a-4d22-ae4e-eb1ae7b77700 admin admin] The token cache is being invalidate because project f239ff88533040a59e227992c5538480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:45:27.341899 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-4702ef3e-ab5a-4d22-ae4e-eb1ae7b77700 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:27.343339 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-4702ef3e-ab5a-4d22-ae4e-eb1ae7b77700 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:27.350617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 870/1773] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:45:26 2022] DELETE /identity/v3/projects/f239ff88533040a59e227992c5538480 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:45:27.360694 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39926fe2-bc82-4ac2-9d24-c0257c6a4f6c None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:27.360694 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39926fe2-bc82-4ac2-9d24-c0257c6a4f6c None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:27.360694 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39926fe2-bc82-4ac2-9d24-c0257c6a4f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:27.384302 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-39926fe2-bc82-4ac2-9d24-c0257c6a4f6c None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:27.385676 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-39926fe2-bc82-4ac2-9d24-c0257c6a4f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:27.427210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 904/1774] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:27 2022] PUT /identity/v3/projects/3b50437225624a7a916a0256179bf9dc/users/8ec088c3a48f42b1af55d10993d676ee/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:27.428668 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bff55-4386-4147-8a2c-e18a5c4b59a8 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:27.428958 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bff55-4386-4147-8a2c-e18a5c4b59a8 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:27.431524 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 871/1775] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:27.432477 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39926fe2-bc82-4ac2-9d24-c0257c6a4f6c None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:27.434016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bff55-4386-4147-8a2c-e18a5c4b59a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:27.440755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-39926fe2-bc82-4ac2-9d24-c0257c6a4f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:27.474582 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39926fe2-bc82-4ac2-9d24-c0257c6a4f6c None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:27.485439 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-d18bff55-4386-4147-8a2c-e18a5c4b59a8 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:27.486699 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-d18bff55-4386-4147-8a2c-e18a5c4b59a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:27.647087 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 905/1776] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:27.653202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bff55-4386-4147-8a2c-e18a5c4b59a8 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:27.663489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c468e2-0839-4813-9c5d-211728ce6019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:27.663795 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c468e2-0839-4813-9c5d-211728ce6019 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:27.664620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c468e2-0839-4813-9c5d-211728ce6019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:27.666479 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c468e2-0839-4813-9c5d-211728ce6019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:27.666761 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bff55-4386-4147-8a2c-e18a5c4b59a8 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:27.669269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c468e2-0839-4813-9c5d-211728ce6019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:27.672173 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c468e2-0839-4813-9c5d-211728ce6019 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:27.672485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198f9f6-8004-482a-a94e-fc6d23dc7085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:27.674439 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198f9f6-8004-482a-a94e-fc6d23dc7085 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:27.674931 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198f9f6-8004-482a-a94e-fc6d23dc7085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:27.675741 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198f9f6-8004-482a-a94e-fc6d23dc7085 admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:27.678954 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0198f9f6-8004-482a-a94e-fc6d23dc7085 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:45:27.679272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0198f9f6-8004-482a-a94e-fc6d23dc7085 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:27.682514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0198f9f6-8004-482a-a94e-fc6d23dc7085 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:27.700178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 872/1777] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:27.703239 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c468e2-0839-4813-9c5d-211728ce6019 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:27.718114 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c468e2-0839-4813-9c5d-211728ce6019 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:27.720244 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 906/1778] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:45:27.725885 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007ec4e-2004-441c-bcaa-61e03eb897fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:27.726194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198f9f6-8004-482a-a94e-fc6d23dc7085 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:27.727384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007ec4e-2004-441c-bcaa-61e03eb897fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:27.727843 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007ec4e-2004-441c-bcaa-61e03eb897fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:27.728245 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007ec4e-2004-441c-bcaa-61e03eb897fa admin admin] PATH_INFO: `/v3/projects/3b50437225624a7a916a0256179bf9dc/users/8ec088c3a48f42b1af55d10993d676ee/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:27.731788 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d007ec4e-2004-441c-bcaa-61e03eb897fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b50437225624a7a916a0256179bf9dc, user_id=8ec088c3a48f42b1af55d10993d676ee, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:27.738214 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d007ec4e-2004-441c-bcaa-61e03eb897fa admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:27.746495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198f9f6-8004-482a-a94e-fc6d23dc7085 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:27.760079 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 873/1779] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:27 2022] PUT /identity/v3/projects/3b50437225624a7a916a0256179bf9dc/users/8ec088c3a48f42b1af55d10993d676ee/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:45:27.775461 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d6b67-c4e7-4121-877b-8eee88f79b37 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:27.775927 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d6b67-c4e7-4121-877b-8eee88f79b37 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:27.776723 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d6b67-c4e7-4121-877b-8eee88f79b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:27.827430 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-c98d6b67-c4e7-4121-877b-8eee88f79b37 None None] MFA Rules not processed for user `8ec088c3a48f42b1af55d10993d676ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:27.828946 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-c98d6b67-c4e7-4121-877b-8eee88f79b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:27.850692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9f20c-d973-4da0-9370-d4475d8667a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:27.851380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9f20c-d973-4da0-9370-d4475d8667a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:27.852584 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9f20c-d973-4da0-9370-d4475d8667a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:27.853836 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9f20c-d973-4da0-9370-d4475d8667a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:27.856497 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab9f20c-d973-4da0-9370-d4475d8667a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:27.857056 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab9f20c-d973-4da0-9370-d4475d8667a2 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:27.986467 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 874/1780] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:27 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:45:28.014749 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 907/1781] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:27 2022] POST /identity/v3/projects => generated 383 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:28.015862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d6b67-c4e7-4121-877b-8eee88f79b37 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:28.032264 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d6b67-c4e7-4121-877b-8eee88f79b37 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:28.097947 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1fed1e-d7d7-4ac4-9d09-ef3ccea75cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:28.098553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fed1e-d7d7-4ac4-9d09-ef3ccea75cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:28.098836 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fed1e-d7d7-4ac4-9d09-ef3ccea75cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:28.099112 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fed1e-d7d7-4ac4-9d09-ef3ccea75cd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:28.099810 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1fed1e-d7d7-4ac4-9d09-ef3ccea75cd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:28.102568 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1fed1e-d7d7-4ac4-9d09-ef3ccea75cd6 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:28.215819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 875/1782] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:28 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:45:28.217546 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9f20c-d973-4da0-9370-d4475d8667a2 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:28.228771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9f20c-d973-4da0-9370-d4475d8667a2 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:28.233801 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee7217-4bf5-497a-bc4d-bd65832919a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:28.234495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee7217-4bf5-497a-bc4d-bd65832919a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:28.234782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee7217-4bf5-497a-bc4d-bd65832919a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:28.235060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee7217-4bf5-497a-bc4d-bd65832919a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:28.235794 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee7217-4bf5-497a-bc4d-bd65832919a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:28.237393 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee7217-4bf5-497a-bc4d-bd65832919a3 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:28.247871 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 908/1783] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:28.252158 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1fed1e-d7d7-4ac4-9d09-ef3ccea75cd6 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:28.262549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1fed1e-d7d7-4ac4-9d09-ef3ccea75cd6 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:28.267922 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967cec0-669d-47b0-b516-bd800aed1261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:28.268622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967cec0-669d-47b0-b516-bd800aed1261 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:28.268907 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967cec0-669d-47b0-b516-bd800aed1261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:28.269189 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967cec0-669d-47b0-b516-bd800aed1261 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:28.269976 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5967cec0-669d-47b0-b516-bd800aed1261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:28.271879 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5967cec0-669d-47b0-b516-bd800aed1261 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:28.288705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:28.321929 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 876/1784] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:28.326451 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee7217-4bf5-497a-bc4d-bd65832919a3 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:28.348378 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee7217-4bf5-497a-bc4d-bd65832919a3 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:28.355336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5a3e3-7927-4f34-8e90-9a65fe234488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:28.356230 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5a3e3-7927-4f34-8e90-9a65fe234488 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:28.358287 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5a3e3-7927-4f34-8e90-9a65fe234488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:28.358287 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5a3e3-7927-4f34-8e90-9a65fe234488 admin admin] PATH_INFO: `/v3/projects/b5f338174f0845baa5a1ed9208cc2ed2/users/0276a2ff5c5e46c59f145c9dfef4600a/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:28.378235 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5a3e3-7927-4f34-8e90-9a65fe234488 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5f338174f0845baa5a1ed9208cc2ed2, user_id=0276a2ff5c5e46c59f145c9dfef4600a, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:28.384076 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5a3e3-7927-4f34-8e90-9a65fe234488 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:28.485794 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:28.491945 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 909/1785] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:28 2022] PUT /identity/v3/projects/b5f338174f0845baa5a1ed9208cc2ed2/users/0276a2ff5c5e46c59f145c9dfef4600a/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:28.510109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849de8b4-cd0f-4fe2-9d79-e9e5b42bad59 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:28.510436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849de8b4-cd0f-4fe2-9d79-e9e5b42bad59 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:28.510713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849de8b4-cd0f-4fe2-9d79-e9e5b42bad59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:28.512108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5a3e3-7927-4f34-8e90-9a65fe234488 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:28.514156 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-e5a5a3e3-7927-4f34-8e90-9a65fe234488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:28.589631 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-849de8b4-cd0f-4fe2-9d79-e9e5b42bad59 None None] MFA Rules not processed for user `0276a2ff5c5e46c59f145c9dfef4600a`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:28.590893 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-849de8b4-cd0f-4fe2-9d79-e9e5b42bad59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:28.610630 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5a3e3-7927-4f34-8e90-9a65fe234488 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:28.625149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114cbc09-1ac6-47e7-8f95-560e6fac88da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:28.626029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cbc09-1ac6-47e7-8f95-560e6fac88da service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:28.627663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cbc09-1ac6-47e7-8f95-560e6fac88da service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:28.629010 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cbc09-1ac6-47e7-8f95-560e6fac88da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:28.632255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-114cbc09-1ac6-47e7-8f95-560e6fac88da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:28.798791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 877/1786] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:28.801092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114cbc09-1ac6-47e7-8f95-560e6fac88da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:28.803300 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849de8b4-cd0f-4fe2-9d79-e9e5b42bad59 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:28.808338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114cbc09-1ac6-47e7-8f95-560e6fac88da service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:28.840348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 910/1787] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:28.937436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849de8b4-cd0f-4fe2-9d79-e9e5b42bad59 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:28.945325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39600f8-e216-4617-adeb-d0a3e759a5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:28.949390 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39600f8-e216-4617-adeb-d0a3e759a5b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:28.949921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39600f8-e216-4617-adeb-d0a3e759a5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:28.950345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39600f8-e216-4617-adeb-d0a3e759a5b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:28.951646 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39600f8-e216-4617-adeb-d0a3e759a5b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:28.954353 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39600f8-e216-4617-adeb-d0a3e759a5b9 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:29.137848 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 878/1788] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:28 2022] POST /identity/v3/projects => generated 381 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:45:29.142268 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114cbc09-1ac6-47e7-8f95-560e6fac88da service cinder] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:29.162676 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114cbc09-1ac6-47e7-8f95-560e6fac88da service cinder] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:29.238924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ecc4d0-538c-47d7-a212-a80c07f6487e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:29.239169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b8080-9383-4d45-b95a-7efbb9f9a75e None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:29.239635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b8080-9383-4d45-b95a-7efbb9f9a75e None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:29.239719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b8080-9383-4d45-b95a-7efbb9f9a75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:29.239956 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ecc4d0-538c-47d7-a212-a80c07f6487e admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:29.241158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ecc4d0-538c-47d7-a212-a80c07f6487e admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:29.241562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ecc4d0-538c-47d7-a212-a80c07f6487e admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:29.242546 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ecc4d0-538c-47d7-a212-a80c07f6487e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:29.245029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ecc4d0-538c-47d7-a212-a80c07f6487e admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:29.263222 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-8d9b8080-9383-4d45-b95a-7efbb9f9a75e None None] MFA Rules not processed for user `58e1f1ac6ea14da0be6e9a24d007847a`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:29.264749 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-8d9b8080-9383-4d45-b95a-7efbb9f9a75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:29.327133 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 879/1789] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Jun 27 18:45:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:29.355704 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 911/1790] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:29 2022] POST /identity/v3/users => generated 297 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:29.358826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b8080-9383-4d45-b95a-7efbb9f9a75e None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:29.371921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b8080-9383-4d45-b95a-7efbb9f9a75e None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:29.376695 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e57172-a4c3-4224-9230-880646a65aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:29.377453 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e57172-a4c3-4224-9230-880646a65aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:29.377742 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e57172-a4c3-4224-9230-880646a65aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:29.378021 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e57172-a4c3-4224-9230-880646a65aa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:29.378895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e57172-a4c3-4224-9230-880646a65aa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:29.380507 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e57172-a4c3-4224-9230-880646a65aa9 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:29.406979 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 880/1791] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:45:29.408210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ecc4d0-538c-47d7-a212-a80c07f6487e admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:29.419235 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ecc4d0-538c-47d7-a212-a80c07f6487e admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:29.423997 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249844c9-7079-4ed2-830f-a0d223ae8254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:29.425002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249844c9-7079-4ed2-830f-a0d223ae8254 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:29.425387 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249844c9-7079-4ed2-830f-a0d223ae8254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:29.425764 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249844c9-7079-4ed2-830f-a0d223ae8254 admin admin] PATH_INFO: `/v3/projects/74eaf84e70f64c37803b49efcd669fb5/users/0baf451b7b28479bbe49030c43f3e6bd/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:29.450143 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249844c9-7079-4ed2-830f-a0d223ae8254 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74eaf84e70f64c37803b49efcd669fb5, user_id=0baf451b7b28479bbe49030c43f3e6bd, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:29.453246 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249844c9-7079-4ed2-830f-a0d223ae8254 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:29.607087 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 912/1792] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:29 2022] PUT /identity/v3/projects/74eaf84e70f64c37803b49efcd669fb5/users/0baf451b7b28479bbe49030c43f3e6bd/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:29.612062 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e57172-a4c3-4224-9230-880646a65aa9 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:29.629718 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e57172-a4c3-4224-9230-880646a65aa9 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:29.708016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249844c9-7079-4ed2-830f-a0d223ae8254 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:29.709854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-249844c9-7079-4ed2-830f-a0d223ae8254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:29.731174 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b1371-9137-49f2-ad9b-734fd77380fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:29.731174 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b1371-9137-49f2-ad9b-734fd77380fd admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:29.731174 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b1371-9137-49f2-ad9b-734fd77380fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:29.731174 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b1371-9137-49f2-ad9b-734fd77380fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:29.732050 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b1371-9137-49f2-ad9b-734fd77380fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:29.736899 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b1371-9137-49f2-ad9b-734fd77380fd admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:29.764789 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 881/1793] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:45:29.767345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b1371-9137-49f2-ad9b-734fd77380fd admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:29.773263 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:29.783192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b1371-9137-49f2-ad9b-734fd77380fd admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:29.818405 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ddff2-2246-426a-b9fe-5f877ad47819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:29.821040 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ddff2-2246-426a-b9fe-5f877ad47819 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:29.821561 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ddff2-2246-426a-b9fe-5f877ad47819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:29.822719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ddff2-2246-426a-b9fe-5f877ad47819 admin admin] PATH_INFO: `/v3/projects/74eaf84e70f64c37803b49efcd669fb5/users/0baf451b7b28479bbe49030c43f3e6bd/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:29.833236 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ddff2-2246-426a-b9fe-5f877ad47819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74eaf84e70f64c37803b49efcd669fb5, user_id=0baf451b7b28479bbe49030c43f3e6bd, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:29.837766 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ddff2-2246-426a-b9fe-5f877ad47819 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:29.873314 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 882/1794] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:29 2022] PUT /identity/v3/projects/74eaf84e70f64c37803b49efcd669fb5/users/0baf451b7b28479bbe49030c43f3e6bd/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:29.875886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2001d-d2c2-4881-b478-3d4160fc95eb None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:29.876183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2001d-d2c2-4881-b478-3d4160fc95eb None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:29.876463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2001d-d2c2-4881-b478-3d4160fc95eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:29.913206 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249844c9-7079-4ed2-830f-a0d223ae8254 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:29.920454 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:29.946154 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-64d2001d-d2c2-4881-b478-3d4160fc95eb None None] MFA Rules not processed for user `0baf451b7b28479bbe49030c43f3e6bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:29.947429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-64d2001d-d2c2-4881-b478-3d4160fc95eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:29.999465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a293d-09eb-44be-9963-8d85f8d9d016 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1529c9a45434a19968e924797688f06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:30.000474 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a293d-09eb-44be-9963-8d85f8d9d016 service swift] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:30.000761 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a293d-09eb-44be-9963-8d85f8d9d016 service swift] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:30.001181 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a293d-09eb-44be-9963-8d85f8d9d016 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:30.111449 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 883/1795] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:29 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:45:30.147483 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a293d-09eb-44be-9963-8d85f8d9d016 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:30.151213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a293d-09eb-44be-9963-8d85f8d9d016 service swift] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:30.168087 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 913/1796] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Jun 27 18:45:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 18:45:30.193488 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e22f4-874e-4308-aa3c-8c54fc654b52 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:30.193854 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e22f4-874e-4308-aa3c-8c54fc654b52 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:30.194113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e22f4-874e-4308-aa3c-8c54fc654b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:30.236812 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-212e22f4-874e-4308-aa3c-8c54fc654b52 None None] MFA Rules not processed for user `dbe621916dbe4f1a8686097fe2bdf762`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:30.239081 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-212e22f4-874e-4308-aa3c-8c54fc654b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:30.253510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a293d-09eb-44be-9963-8d85f8d9d016 service swift] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:30.255472 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-e53a293d-09eb-44be-9963-8d85f8d9d016 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:30.429849 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a293d-09eb-44be-9963-8d85f8d9d016 service swift] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:30.448958 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e64ea6-f2e7-48fd-a50e-d7a481cae9c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:30.449624 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e64ea6-f2e7-48fd-a50e-d7a481cae9c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:30.449905 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e64ea6-f2e7-48fd-a50e-d7a481cae9c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:30.450181 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e64ea6-f2e7-48fd-a50e-d7a481cae9c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:30.458414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 884/1797] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Jun 27 18:45:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:30.476216 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e64ea6-f2e7-48fd-a50e-d7a481cae9c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:30.495960 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e64ea6-f2e7-48fd-a50e-d7a481cae9c8 service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:30.574147 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 914/1798] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:30.799484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e22f4-874e-4308-aa3c-8c54fc654b52 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:30.801275 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-212e22f4-874e-4308-aa3c-8c54fc654b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:30.817958 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e22f4-874e-4308-aa3c-8c54fc654b52 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:30.830528 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49a49d-a76b-4300-b1ed-5fd65fe10256 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:30.831375 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49a49d-a76b-4300-b1ed-5fd65fe10256 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:30.831629 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49a49d-a76b-4300-b1ed-5fd65fe10256 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:30.831907 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49a49d-a76b-4300-b1ed-5fd65fe10256 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:30.844490 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e64ea6-f2e7-48fd-a50e-d7a481cae9c8 service cinder] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:30.861613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-14e64ea6-f2e7-48fd-a50e-d7a481cae9c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:30.870983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e64ea6-f2e7-48fd-a50e-d7a481cae9c8 service cinder] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:30.876403 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e49a49d-a76b-4300-b1ed-5fd65fe10256 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:30.877783 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e49a49d-a76b-4300-b1ed-5fd65fe10256 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:30.890336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c51765-0553-49ca-9b9d-37617699eea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:30.891282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c51765-0553-49ca-9b9d-37617699eea6 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:30.891503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c51765-0553-49ca-9b9d-37617699eea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:30.894473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c51765-0553-49ca-9b9d-37617699eea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:30.906325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 885/1799] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:30.911258 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c51765-0553-49ca-9b9d-37617699eea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:30.912702 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c51765-0553-49ca-9b9d-37617699eea6 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:30.934561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 915/1800] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:45:31.580111 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:31.970196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:32.436450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49a49d-a76b-4300-b1ed-5fd65fe10256 service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:32.459812 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49a49d-a76b-4300-b1ed-5fd65fe10256 service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:32.475346 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3c1a2-6ff9-44fa-929b-9c53c0ec16fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:32.485140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3c1a2-6ff9-44fa-929b-9c53c0ec16fa service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:32.485564 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3c1a2-6ff9-44fa-929b-9c53c0ec16fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:32.485971 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3c1a2-6ff9-44fa-929b-9c53c0ec16fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:32.717573 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3c1a2-6ff9-44fa-929b-9c53c0ec16fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:32.719229 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3c1a2-6ff9-44fa-929b-9c53c0ec16fa service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:32.751876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 886/1801] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:32.909299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a77a8f-e72f-4416-86be-81c75b54ea5f None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:32.910948 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a77a8f-e72f-4416-86be-81c75b54ea5f None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:32.911424 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a77a8f-e72f-4416-86be-81c75b54ea5f None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:32.914297 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 916/1802] 10.222.0.15 () {54 vars in 921 bytes} [Mon Jun 27 18:45:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:45:32.928370 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737c35a-7b6b-465c-b8af-6ede29b7d2ce None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:32.928944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737c35a-7b6b-465c-b8af-6ede29b7d2ce None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:32.930331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737c35a-7b6b-465c-b8af-6ede29b7d2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:32.989901 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-5737c35a-7b6b-465c-b8af-6ede29b7d2ce None None] MFA Rules not processed for user `1dddd4efb44c4152bb4151f7715f2223`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:32.991172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-5737c35a-7b6b-465c-b8af-6ede29b7d2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:33.149973 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 887/1803] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Jun 27 18:45:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:45:33.238031 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a77a8f-e72f-4416-86be-81c75b54ea5f None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:33.239954 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-62a77a8f-e72f-4416-86be-81c75b54ea5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:33.259394 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a77a8f-e72f-4416-86be-81c75b54ea5f None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:33.265820 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a86b4-f6ca-41c2-bf0e-600cd94c4036 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:33.266599 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a86b4-f6ca-41c2-bf0e-600cd94c4036 service nova] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:33.267051 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a86b4-f6ca-41c2-bf0e-600cd94c4036 service nova] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:33.267326 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a86b4-f6ca-41c2-bf0e-600cd94c4036 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:33.278915 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a86b4-f6ca-41c2-bf0e-600cd94c4036 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:33.280726 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a86b4-f6ca-41c2-bf0e-600cd94c4036 service nova] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:33.292260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 917/1804] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:33.469098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78bd2d-187e-4428-ac0e-c87b3ee8d431 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:33.469393 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78bd2d-187e-4428-ac0e-c87b3ee8d431 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:33.469672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78bd2d-187e-4428-ac0e-c87b3ee8d431 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:33.471377 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 888/1805] 10.222.0.15 () {54 vars in 921 bytes} [Mon Jun 27 18:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:45:33.479590 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627420e-9388-44a0-9001-31fd23d98840 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:33.479826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627420e-9388-44a0-9001-31fd23d98840 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:33.480049 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627420e-9388-44a0-9001-31fd23d98840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:33.510859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-8627420e-9388-44a0-9001-31fd23d98840 None None] MFA Rules not processed for user `1dddd4efb44c4152bb4151f7715f2223`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:33.513083 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-8627420e-9388-44a0-9001-31fd23d98840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:33.531448 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 918/1806] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Jun 27 18:45:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:33.557420 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78bd2d-187e-4428-ac0e-c87b3ee8d431 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:33.576180 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:33.593149 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78bd2d-187e-4428-ac0e-c87b3ee8d431 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:33.616677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6831e9f-7bb8-4c5d-a468-ee76e515fa72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:33.616677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6831e9f-7bb8-4c5d-a468-ee76e515fa72 service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:33.616677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6831e9f-7bb8-4c5d-a468-ee76e515fa72 service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:33.616677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6831e9f-7bb8-4c5d-a468-ee76e515fa72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:33.636977 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6831e9f-7bb8-4c5d-a468-ee76e515fa72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:33.640291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6831e9f-7bb8-4c5d-a468-ee76e515fa72 service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:33.681207 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:33.693044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 889/1807] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:34.907373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:34.941555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:35.243658 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:35.604380 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:35.819781 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:36.296642 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:36.366790 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:36.873905 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:36.941764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:37.058109 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627420e-9388-44a0-9001-31fd23d98840 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:37.067937 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627420e-9388-44a0-9001-31fd23d98840 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:37.072966 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e9eb4-ab1c-403a-9156-40e0d713483f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:37.073807 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e9eb4-ab1c-403a-9156-40e0d713483f service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:37.074228 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e9eb4-ab1c-403a-9156-40e0d713483f service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:37.074689 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e9eb4-ab1c-403a-9156-40e0d713483f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:37.170963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e9eb4-ab1c-403a-9156-40e0d713483f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:37.172750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e9eb4-ab1c-403a-9156-40e0d713483f service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:37.186646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 919/1808] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:37 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:37.358940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:38.214894 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6831e9f-7bb8-4c5d-a468-ee76e515fa72 service nova] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:38.216680 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-b6831e9f-7bb8-4c5d-a468-ee76e515fa72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:38.342246 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6831e9f-7bb8-4c5d-a468-ee76e515fa72 service nova] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:38.347682 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54dec9-0714-4e00-9096-4f778a73008e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:38.348572 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54dec9-0714-4e00-9096-4f778a73008e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:38.348854 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54dec9-0714-4e00-9096-4f778a73008e admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:38.349171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54dec9-0714-4e00-9096-4f778a73008e admin admin] PATH_INFO: `/v3/users/d42d132ffa6d4744b20415827926dcbf` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:38.350617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54dec9-0714-4e00-9096-4f778a73008e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d42d132ffa6d4744b20415827926dcbf)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:38.353758 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54dec9-0714-4e00-9096-4f778a73008e admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:38.618994 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-bf54dec9-0714-4e00-9096-4f778a73008e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd42d132ffa6d4744b20415827926dcbf'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:38.626688 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-bf54dec9-0714-4e00-9096-4f778a73008e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd42d132ffa6d4744b20415827926dcbf'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:38.627639 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-bf54dec9-0714-4e00-9096-4f778a73008e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd42d132ffa6d4744b20415827926dcbf'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:38.665071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:38.669020 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-bf54dec9-0714-4e00-9096-4f778a73008e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd42d132ffa6d4744b20415827926dcbf'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:38.724678 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-bf54dec9-0714-4e00-9096-4f778a73008e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd42d132ffa6d4744b20415827926dcbf'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:38.759218 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 890/1809] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:45:38 2022] DELETE /identity/v3/users/d42d132ffa6d4744b20415827926dcbf => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:45:38.766521 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29351b9f-513e-4f5c-a2c0-47f22af94ef2 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:38.767953 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29351b9f-513e-4f5c-a2c0-47f22af94ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:38.768444 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29351b9f-513e-4f5c-a2c0-47f22af94ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:38.790117 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-29351b9f-513e-4f5c-a2c0-47f22af94ef2 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:38.792346 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-29351b9f-513e-4f5c-a2c0-47f22af94ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:38.892735 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54dec9-0714-4e00-9096-4f778a73008e admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:38.895070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-bf54dec9-0714-4e00-9096-4f778a73008e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:38.909917 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 920/1810] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:38.913275 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54dec9-0714-4e00-9096-4f778a73008e admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:38.921770 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29351b9f-513e-4f5c-a2c0-47f22af94ef2 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:38.922498 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38742e2-bb4e-49af-9a9c-e1835ca71a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:38.923415 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38742e2-bb4e-49af-9a9c-e1835ca71a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:38.923759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38742e2-bb4e-49af-9a9c-e1835ca71a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:38.924033 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38742e2-bb4e-49af-9a9c-e1835ca71a4c admin admin] PATH_INFO: `/v3/users/0276a2ff5c5e46c59f145c9dfef4600a` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:38.925447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38742e2-bb4e-49af-9a9c-e1835ca71a4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0276a2ff5c5e46c59f145c9dfef4600a)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:38.926178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-29351b9f-513e-4f5c-a2c0-47f22af94ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:38.927584 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38742e2-bb4e-49af-9a9c-e1835ca71a4c admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:39.050944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29351b9f-513e-4f5c-a2c0-47f22af94ef2 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:39.055518 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89bd397-d148-49d1-ad96-02aedd314711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:39.056194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bd397-d148-49d1-ad96-02aedd314711 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:39.056478 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bd397-d148-49d1-ad96-02aedd314711 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:39.057289 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bd397-d148-49d1-ad96-02aedd314711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:39.087789 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89bd397-d148-49d1-ad96-02aedd314711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:39.089361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89bd397-d148-49d1-ad96-02aedd314711 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:39.109290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 921/1811] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:39.175518 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c38742e2-bb4e-49af-9a9c-e1835ca71a4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0276a2ff5c5e46c59f145c9dfef4600a'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:39.184274 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c38742e2-bb4e-49af-9a9c-e1835ca71a4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0276a2ff5c5e46c59f145c9dfef4600a'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:39.185339 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c38742e2-bb4e-49af-9a9c-e1835ca71a4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0276a2ff5c5e46c59f145c9dfef4600a'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:39.203353 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c38742e2-bb4e-49af-9a9c-e1835ca71a4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0276a2ff5c5e46c59f145c9dfef4600a'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:39.309051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c38742e2-bb4e-49af-9a9c-e1835ca71a4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0276a2ff5c5e46c59f145c9dfef4600a'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:39.318279 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 891/1812] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:45:38 2022] DELETE /identity/v3/users/0276a2ff5c5e46c59f145c9dfef4600a => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:45:39.324230 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a4f35-8222-4fed-ac7b-4522ab3335d3 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:39.324523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a4f35-8222-4fed-ac7b-4522ab3335d3 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:39.324866 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a4f35-8222-4fed-ac7b-4522ab3335d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:39.359000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-763a4f35-8222-4fed-ac7b-4522ab3335d3 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:39.361192 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-763a4f35-8222-4fed-ac7b-4522ab3335d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:39.410688 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:39.530563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 922/1813] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:45:39.543649 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38742e2-bb4e-49af-9a9c-e1835ca71a4c admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:39.545836 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-c38742e2-bb4e-49af-9a9c-e1835ca71a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:39.640914 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a4f35-8222-4fed-ac7b-4522ab3335d3 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:39.642603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-763a4f35-8222-4fed-ac7b-4522ab3335d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:39.653691 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38742e2-bb4e-49af-9a9c-e1835ca71a4c admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:39.658326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad5b1fd-0b4b-4b3e-9cce-0bd6f80d0ffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:39.659103 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5b1fd-0b4b-4b3e-9cce-0bd6f80d0ffe service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:39.659385 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5b1fd-0b4b-4b3e-9cce-0bd6f80d0ffe service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:39.659669 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5b1fd-0b4b-4b3e-9cce-0bd6f80d0ffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:39.672867 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a4f35-8222-4fed-ac7b-4522ab3335d3 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:39.674110 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5b1fd-0b4b-4b3e-9cce-0bd6f80d0ffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:39.676257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5b1fd-0b4b-4b3e-9cce-0bd6f80d0ffe service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:39.684810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3b0ec-6f17-4fe9-9440-aa78b2b05121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:39.686473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3b0ec-6f17-4fe9-9440-aa78b2b05121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:39.690641 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 892/1814] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:39.697251 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad5b1fd-0b4b-4b3e-9cce-0bd6f80d0ffe service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:39.699186 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-cad5b1fd-0b4b-4b3e-9cce-0bd6f80d0ffe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:39.700370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3b0ec-6f17-4fe9-9440-aa78b2b05121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:39.703273 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3b0ec-6f17-4fe9-9440-aa78b2b05121 admin admin] PATH_INFO: `/v3/users/120aa1da5ea0447bab7d3d27d808e581` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:39.705494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df3b0ec-6f17-4fe9-9440-aa78b2b05121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=120aa1da5ea0447bab7d3d27d808e581)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:39.715798 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad5b1fd-0b4b-4b3e-9cce-0bd6f80d0ffe service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:39.715927 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df3b0ec-6f17-4fe9-9440-aa78b2b05121 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:39.740407 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0fe99-88e4-466f-bb8c-60b0afafa453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:39.741254 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0fe99-88e4-466f-bb8c-60b0afafa453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:39.741464 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0fe99-88e4-466f-bb8c-60b0afafa453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:39.742315 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0fe99-88e4-466f-bb8c-60b0afafa453 admin admin] PATH_INFO: `/v3/projects/6fd170766ea74cd88a87acfbd1687a05` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:39.745493 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0fe99-88e4-466f-bb8c-60b0afafa453 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fd170766ea74cd88a87acfbd1687a05)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:39.748103 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0fe99-88e4-466f-bb8c-60b0afafa453 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:39.947370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-4df3b0ec-6f17-4fe9-9440-aa78b2b05121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '120aa1da5ea0447bab7d3d27d808e581'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:39.947984 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-5cf0fe99-88e4-466f-bb8c-60b0afafa453 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fd170766ea74cd88a87acfbd1687a05', 'request_id': 'req-5cf0fe99-88e4-466f-bb8c-60b0afafa453'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:39.954289 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-4df3b0ec-6f17-4fe9-9440-aa78b2b05121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '120aa1da5ea0447bab7d3d27d808e581'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:39.955225 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-4df3b0ec-6f17-4fe9-9440-aa78b2b05121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '120aa1da5ea0447bab7d3d27d808e581'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:39.955829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-4df3b0ec-6f17-4fe9-9440-aa78b2b05121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '120aa1da5ea0447bab7d3d27d808e581'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:39.967848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-4df3b0ec-6f17-4fe9-9440-aa78b2b05121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '120aa1da5ea0447bab7d3d27d808e581'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:40.013996 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-5cf0fe99-88e4-466f-bb8c-60b0afafa453 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fd170766ea74cd88a87acfbd1687a05', 'request_id': 'req-5cf0fe99-88e4-466f-bb8c-60b0afafa453'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:40.016612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=81,keystone:INSERT=3,keystone:DELETE=16 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:40.021237 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-5cf0fe99-88e4-466f-bb8c-60b0afafa453 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fd170766ea74cd88a87acfbd1687a05', 'request_id': 'req-5cf0fe99-88e4-466f-bb8c-60b0afafa453'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:40.039282 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-5cf0fe99-88e4-466f-bb8c-60b0afafa453 admin admin] The token cache is being invalidate because project 6fd170766ea74cd88a87acfbd1687a05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:45:40.040012 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-5cf0fe99-88e4-466f-bb8c-60b0afafa453 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:40.041018 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-5cf0fe99-88e4-466f-bb8c-60b0afafa453 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:40.044173 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 893/1815] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:45:39 2022] DELETE /identity/v3/projects/6fd170766ea74cd88a87acfbd1687a05 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:45:40.048164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0fe99-88e4-466f-bb8c-60b0afafa453 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:40.049740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-5cf0fe99-88e4-466f-bb8c-60b0afafa453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:40.079631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 923/1816] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:45:39 2022] DELETE /identity/v3/users/120aa1da5ea0447bab7d3d27d808e581 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:45:40.084535 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3805a4-c3bf-4e2d-afd5-11cdb096e45a None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:40.084900 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3805a4-c3bf-4e2d-afd5-11cdb096e45a None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:40.086068 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3805a4-c3bf-4e2d-afd5-11cdb096e45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:40.142429 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-ea3805a4-c3bf-4e2d-afd5-11cdb096e45a None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:40.143951 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ea3805a4-c3bf-4e2d-afd5-11cdb096e45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:40.151970 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0fe99-88e4-466f-bb8c-60b0afafa453 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:40.277432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d92005-ce1a-4640-b525-9bf9c124fedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:40.278991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d92005-ce1a-4640-b525-9bf9c124fedb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:40.279276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d92005-ce1a-4640-b525-9bf9c124fedb admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:40.279554 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d92005-ce1a-4640-b525-9bf9c124fedb admin admin] PATH_INFO: `/v3/users/f341e7a88d144b52aabecc3887c6ba0b` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:40.281005 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d92005-ce1a-4640-b525-9bf9c124fedb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f341e7a88d144b52aabecc3887c6ba0b)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:40.283134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d92005-ce1a-4640-b525-9bf9c124fedb admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:40.294685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 924/1817] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:45:40.301631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3805a4-c3bf-4e2d-afd5-11cdb096e45a None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:40.303171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ea3805a4-c3bf-4e2d-afd5-11cdb096e45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:40.522326 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:40.556939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3805a4-c3bf-4e2d-afd5-11cdb096e45a None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:40.562358 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a862f1-87c4-4b25-b5c6-778a07b08610 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:40.563109 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a862f1-87c4-4b25-b5c6-778a07b08610 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:40.563394 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a862f1-87c4-4b25-b5c6-778a07b08610 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:40.563722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a862f1-87c4-4b25-b5c6-778a07b08610 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:40.575029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a862f1-87c4-4b25-b5c6-778a07b08610 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:40.578741 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a862f1-87c4-4b25-b5c6-778a07b08610 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:40.602488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 925/1818] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:40.605409 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a862f1-87c4-4b25-b5c6-778a07b08610 service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:40.608335 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-b3a862f1-87c4-4b25-b5c6-778a07b08610 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:40.609368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f6d92005-ce1a-4640-b525-9bf9c124fedb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f341e7a88d144b52aabecc3887c6ba0b'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:40.623826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f6d92005-ce1a-4640-b525-9bf9c124fedb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f341e7a88d144b52aabecc3887c6ba0b'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:40.624732 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f6d92005-ce1a-4640-b525-9bf9c124fedb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f341e7a88d144b52aabecc3887c6ba0b'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:40.642538 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a862f1-87c4-4b25-b5c6-778a07b08610 service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:40.649210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae4f2c-fbe8-4cb8-b7c7-a4cae03d7f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:40.649839 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae4f2c-fbe8-4cb8-b7c7-a4cae03d7f18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:40.650110 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae4f2c-fbe8-4cb8-b7c7-a4cae03d7f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:40.650422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae4f2c-fbe8-4cb8-b7c7-a4cae03d7f18 admin admin] PATH_INFO: `/v3/projects/b5f338174f0845baa5a1ed9208cc2ed2` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:40.651758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae4f2c-fbe8-4cb8-b7c7-a4cae03d7f18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5f338174f0845baa5a1ed9208cc2ed2)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:40.654192 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae4f2c-fbe8-4cb8-b7c7-a4cae03d7f18 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:40.656231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f6d92005-ce1a-4640-b525-9bf9c124fedb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f341e7a88d144b52aabecc3887c6ba0b'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:40.713546 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f6d92005-ce1a-4640-b525-9bf9c124fedb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f341e7a88d144b52aabecc3887c6ba0b'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:40.721417 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 894/1819] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:45:40 2022] DELETE /identity/v3/users/f341e7a88d144b52aabecc3887c6ba0b => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:40.778134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-24ae4f2c-fbe8-4cb8-b7c7-a4cae03d7f18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5f338174f0845baa5a1ed9208cc2ed2', 'request_id': 'req-24ae4f2c-fbe8-4cb8-b7c7-a4cae03d7f18'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:40.873198 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-24ae4f2c-fbe8-4cb8-b7c7-a4cae03d7f18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5f338174f0845baa5a1ed9208cc2ed2', 'request_id': 'req-24ae4f2c-fbe8-4cb8-b7c7-a4cae03d7f18'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:40.880360 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-24ae4f2c-fbe8-4cb8-b7c7-a4cae03d7f18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5f338174f0845baa5a1ed9208cc2ed2', 'request_id': 'req-24ae4f2c-fbe8-4cb8-b7c7-a4cae03d7f18'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:40.906003 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-24ae4f2c-fbe8-4cb8-b7c7-a4cae03d7f18 admin admin] The token cache is being invalidate because project b5f338174f0845baa5a1ed9208cc2ed2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:45:40.906294 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-24ae4f2c-fbe8-4cb8-b7c7-a4cae03d7f18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:40.907039 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-24ae4f2c-fbe8-4cb8-b7c7-a4cae03d7f18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:40.908713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 926/1820] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:45:40 2022] DELETE /identity/v3/projects/b5f338174f0845baa5a1ed9208cc2ed2 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:40.914246 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d92005-ce1a-4640-b525-9bf9c124fedb admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:40.922879 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f6d92005-ce1a-4640-b525-9bf9c124fedb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:41.027891 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d92005-ce1a-4640-b525-9bf9c124fedb admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:41.084810 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cc5c7-e25d-44b7-89e0-56ff674acf9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:41.085984 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cc5c7-e25d-44b7-89e0-56ff674acf9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:41.086268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cc5c7-e25d-44b7-89e0-56ff674acf9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:41.086574 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cc5c7-e25d-44b7-89e0-56ff674acf9e admin admin] PATH_INFO: `/v3/users/0baf451b7b28479bbe49030c43f3e6bd` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:41.089018 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1cc5c7-e25d-44b7-89e0-56ff674acf9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0baf451b7b28479bbe49030c43f3e6bd)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:41.089807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1cc5c7-e25d-44b7-89e0-56ff674acf9e admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:41.142604 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae4f2c-fbe8-4cb8-b7c7-a4cae03d7f18 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:41.146456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-24ae4f2c-fbe8-4cb8-b7c7-a4cae03d7f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:41.146603 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:41.157124 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae4f2c-fbe8-4cb8-b7c7-a4cae03d7f18 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:41.161813 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0507c29-f644-49a6-b2d6-2182133c4e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:41.162731 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0507c29-f644-49a6-b2d6-2182133c4e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:41.163013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0507c29-f644-49a6-b2d6-2182133c4e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:41.163950 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0507c29-f644-49a6-b2d6-2182133c4e6c admin admin] PATH_INFO: `/v3/projects/6509d0e8c91a4d86b6f58a0feb725064` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:41.165791 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0507c29-f644-49a6-b2d6-2182133c4e6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6509d0e8c91a4d86b6f58a0feb725064)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:41.173614 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0507c29-f644-49a6-b2d6-2182133c4e6c admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:41.369871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-cc1cc5c7-e25d-44b7-89e0-56ff674acf9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0baf451b7b28479bbe49030c43f3e6bd'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:41.373677 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b0507c29-f644-49a6-b2d6-2182133c4e6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6509d0e8c91a4d86b6f58a0feb725064', 'request_id': 'req-b0507c29-f644-49a6-b2d6-2182133c4e6c'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:41.380246 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-cc1cc5c7-e25d-44b7-89e0-56ff674acf9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0baf451b7b28479bbe49030c43f3e6bd'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:41.381122 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-cc1cc5c7-e25d-44b7-89e0-56ff674acf9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0baf451b7b28479bbe49030c43f3e6bd'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:41.393270 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-cc1cc5c7-e25d-44b7-89e0-56ff674acf9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0baf451b7b28479bbe49030c43f3e6bd'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:41.461397 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b0507c29-f644-49a6-b2d6-2182133c4e6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6509d0e8c91a4d86b6f58a0feb725064', 'request_id': 'req-b0507c29-f644-49a6-b2d6-2182133c4e6c'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:41.468993 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b0507c29-f644-49a6-b2d6-2182133c4e6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6509d0e8c91a4d86b6f58a0feb725064', 'request_id': 'req-b0507c29-f644-49a6-b2d6-2182133c4e6c'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:41.480420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b0507c29-f644-49a6-b2d6-2182133c4e6c admin admin] The token cache is being invalidate because project 6509d0e8c91a4d86b6f58a0feb725064 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:45:41.480708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b0507c29-f644-49a6-b2d6-2182133c4e6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:41.481523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b0507c29-f644-49a6-b2d6-2182133c4e6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:41.483560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 927/1821] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:45:41 2022] DELETE /identity/v3/projects/6509d0e8c91a4d86b6f58a0feb725064 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:45:41.484305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0507c29-f644-49a6-b2d6-2182133c4e6c admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:41.485554 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-b0507c29-f644-49a6-b2d6-2182133c4e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:41.530168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-cc1cc5c7-e25d-44b7-89e0-56ff674acf9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0baf451b7b28479bbe49030c43f3e6bd'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:41.536638 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 895/1822] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:45:40 2022] DELETE /identity/v3/users/0baf451b7b28479bbe49030c43f3e6bd => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:41.537878 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68027a19-512c-4bc2-a1c1-26698ed1ce56 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:41.538160 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68027a19-512c-4bc2-a1c1-26698ed1ce56 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:41.539786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68027a19-512c-4bc2-a1c1-26698ed1ce56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:41.556056 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0507c29-f644-49a6-b2d6-2182133c4e6c admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:41.569930 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-68027a19-512c-4bc2-a1c1-26698ed1ce56 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:41.573521 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-68027a19-512c-4bc2-a1c1-26698ed1ce56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:41.651596 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 896/1823] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:41.662587 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b720bf-c99a-4700-a900-3ce7b4d6fc5f None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:41.662887 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b720bf-c99a-4700-a900-3ce7b4d6fc5f None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:41.663166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b720bf-c99a-4700-a900-3ce7b4d6fc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:41.663821 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc711ab-9225-4783-b2cf-3c7f8bfc9b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:41.664688 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc711ab-9225-4783-b2cf-3c7f8bfc9b44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:41.665080 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc711ab-9225-4783-b2cf-3c7f8bfc9b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:41.665332 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc711ab-9225-4783-b2cf-3c7f8bfc9b44 admin admin] PATH_INFO: `/v3/projects/7981a4af2112498dae3233268827f781` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:41.666561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc711ab-9225-4783-b2cf-3c7f8bfc9b44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7981a4af2112498dae3233268827f781)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:41.668527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc711ab-9225-4783-b2cf-3c7f8bfc9b44 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:41.685901 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-d9b720bf-c99a-4700-a900-3ce7b4d6fc5f None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:41.687236 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-d9b720bf-c99a-4700-a900-3ce7b4d6fc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:41.698121 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 897/1824] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:41.702294 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b720bf-c99a-4700-a900-3ce7b4d6fc5f None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:41.712698 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b720bf-c99a-4700-a900-3ce7b4d6fc5f None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:41.713351 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:41.717542 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21729f-890c-4571-81dd-ddbe1c78d8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:41.718348 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21729f-890c-4571-81dd-ddbe1c78d8a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:41.718722 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21729f-890c-4571-81dd-ddbe1c78d8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:41.719059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21729f-890c-4571-81dd-ddbe1c78d8a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:41.719853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21729f-890c-4571-81dd-ddbe1c78d8a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:45:41.720223 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21729f-890c-4571-81dd-ddbe1c78d8a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:41.722419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21729f-890c-4571-81dd-ddbe1c78d8a7 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:41.736327 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 898/1825] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:45:41.740297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21729f-890c-4571-81dd-ddbe1c78d8a7 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:41.752375 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21729f-890c-4571-81dd-ddbe1c78d8a7 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:41.758285 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba54167-44ef-4fc6-89e2-2c5b5b312126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:41.759349 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba54167-44ef-4fc6-89e2-2c5b5b312126 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:41.759712 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba54167-44ef-4fc6-89e2-2c5b5b312126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:41.760178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba54167-44ef-4fc6-89e2-2c5b5b312126 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:41.761378 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba54167-44ef-4fc6-89e2-2c5b5b312126 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:41.764603 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba54167-44ef-4fc6-89e2-2c5b5b312126 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:41.777272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-bbc711ab-9225-4783-b2cf-3c7f8bfc9b44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7981a4af2112498dae3233268827f781', 'request_id': 'req-bbc711ab-9225-4783-b2cf-3c7f8bfc9b44'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:41.842034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-bbc711ab-9225-4783-b2cf-3c7f8bfc9b44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7981a4af2112498dae3233268827f781', 'request_id': 'req-bbc711ab-9225-4783-b2cf-3c7f8bfc9b44'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:41.849774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-bbc711ab-9225-4783-b2cf-3c7f8bfc9b44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7981a4af2112498dae3233268827f781', 'request_id': 'req-bbc711ab-9225-4783-b2cf-3c7f8bfc9b44'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:41.854774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 899/1826] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:41 2022] POST /identity/v3/projects => generated 405 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:45:41.856305 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba54167-44ef-4fc6-89e2-2c5b5b312126 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:41.863898 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-bbc711ab-9225-4783-b2cf-3c7f8bfc9b44 admin admin] The token cache is being invalidate because project 7981a4af2112498dae3233268827f781 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:45:41.864197 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-bbc711ab-9225-4783-b2cf-3c7f8bfc9b44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:41.864903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-bbc711ab-9225-4783-b2cf-3c7f8bfc9b44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:41.867991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 928/1827] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:45:41 2022] DELETE /identity/v3/projects/7981a4af2112498dae3233268827f781 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:41.872683 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee635844-971c-4c8c-aade-ae701b10745e None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:41.873077 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee635844-971c-4c8c-aade-ae701b10745e None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:41.873361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee635844-971c-4c8c-aade-ae701b10745e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:41.895213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-ee635844-971c-4c8c-aade-ae701b10745e None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:41.896490 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ee635844-971c-4c8c-aade-ae701b10745e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:41.964735 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba54167-44ef-4fc6-89e2-2c5b5b312126 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:41.972423 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e376c5f-98a2-42a0-ae9b-60a778d527ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:41.973213 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e376c5f-98a2-42a0-ae9b-60a778d527ef admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:41.973374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e376c5f-98a2-42a0-ae9b-60a778d527ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:41.974166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e376c5f-98a2-42a0-ae9b-60a778d527ef admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:41.974744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e376c5f-98a2-42a0-ae9b-60a778d527ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:41.977160 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e376c5f-98a2-42a0-ae9b-60a778d527ef admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:41.995513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 929/1828] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:42.003261 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10fdfd-5039-4311-ae6c-8a2e77f1c2f4 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:42.003693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10fdfd-5039-4311-ae6c-8a2e77f1c2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:42.004275 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10fdfd-5039-4311-ae6c-8a2e77f1c2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:42.029293 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-fd10fdfd-5039-4311-ae6c-8a2e77f1c2f4 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:42.030565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-fd10fdfd-5039-4311-ae6c-8a2e77f1c2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:42.048457 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 930/1829] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:42.049563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10fdfd-5039-4311-ae6c-8a2e77f1c2f4 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:42.060198 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10fdfd-5039-4311-ae6c-8a2e77f1c2f4 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:42.064763 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fbb63-f350-4379-a7c2-5eae3830c0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:42.065670 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fbb63-f350-4379-a7c2-5eae3830c0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:42.065991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fbb63-f350-4379-a7c2-5eae3830c0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:42.066227 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fbb63-f350-4379-a7c2-5eae3830c0a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:42.067029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1fbb63-f350-4379-a7c2-5eae3830c0a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:45:42.067334 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1fbb63-f350-4379-a7c2-5eae3830c0a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:42.068931 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1fbb63-f350-4379-a7c2-5eae3830c0a4 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:42.081310 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 931/1830] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:45:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:45:42.086297 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fbb63-f350-4379-a7c2-5eae3830c0a4 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:42.096019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fbb63-f350-4379-a7c2-5eae3830c0a4 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:42.101403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6dd9fc-b75c-4cde-8f0d-984ec3dd1481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:42.102752 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6dd9fc-b75c-4cde-8f0d-984ec3dd1481 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:42.103042 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6dd9fc-b75c-4cde-8f0d-984ec3dd1481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:42.103316 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6dd9fc-b75c-4cde-8f0d-984ec3dd1481 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:42.104004 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6dd9fc-b75c-4cde-8f0d-984ec3dd1481 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:42.105903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6dd9fc-b75c-4cde-8f0d-984ec3dd1481 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:42.112473 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 900/1831] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:41 2022] POST /identity/v3/users => generated 303 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:42.118860 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e376c5f-98a2-42a0-ae9b-60a778d527ef admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:42.120783 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-6e376c5f-98a2-42a0-ae9b-60a778d527ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:42.146028 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e376c5f-98a2-42a0-ae9b-60a778d527ef admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:42.146028 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b885221-9557-4bf9-b649-99ea3480aae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:42.146028 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b885221-9557-4bf9-b649-99ea3480aae2 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:42.146028 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b885221-9557-4bf9-b649-99ea3480aae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:42.147126 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b885221-9557-4bf9-b649-99ea3480aae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:42.147126 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b885221-9557-4bf9-b649-99ea3480aae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:42.147126 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b885221-9557-4bf9-b649-99ea3480aae2 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:42.152560 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 901/1832] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:45:42.156549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b885221-9557-4bf9-b649-99ea3480aae2 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:42.167664 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:42.173804 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b885221-9557-4bf9-b649-99ea3480aae2 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:42.182808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0db5f-b9cc-4694-8d27-8dcb3507182b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:42.183567 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0db5f-b9cc-4694-8d27-8dcb3507182b admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:42.183853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0db5f-b9cc-4694-8d27-8dcb3507182b admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:42.184209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0db5f-b9cc-4694-8d27-8dcb3507182b admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:42.185793 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0db5f-b9cc-4694-8d27-8dcb3507182b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:42.187477 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0db5f-b9cc-4694-8d27-8dcb3507182b admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:42.205835 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 902/1833] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:42 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:45:42.205835 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0db5f-b9cc-4694-8d27-8dcb3507182b admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:42.205835 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-4af0db5f-b9cc-4694-8d27-8dcb3507182b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:42.231763 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0db5f-b9cc-4694-8d27-8dcb3507182b admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:42.235557 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 932/1834] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:42 2022] POST /identity/v3/projects => generated 367 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:42.237000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6dd9fc-b75c-4cde-8f0d-984ec3dd1481 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:42.241087 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca24d62-9959-4e79-9c97-cdcbc7700671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:42.243202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca24d62-9959-4e79-9c97-cdcbc7700671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:42.243557 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca24d62-9959-4e79-9c97-cdcbc7700671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:42.243908 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca24d62-9959-4e79-9c97-cdcbc7700671 admin admin] PATH_INFO: `/v3/projects/74eaf84e70f64c37803b49efcd669fb5` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:42.245389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca24d62-9959-4e79-9c97-cdcbc7700671 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74eaf84e70f64c37803b49efcd669fb5)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:42.247424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca24d62-9959-4e79-9c97-cdcbc7700671 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:42.329722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6dd9fc-b75c-4cde-8f0d-984ec3dd1481 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:42.336619 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafff55-c642-4010-9494-35ab6ebf2281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:42.337355 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafff55-c642-4010-9494-35ab6ebf2281 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:42.337746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafff55-c642-4010-9494-35ab6ebf2281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:42.338035 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafff55-c642-4010-9494-35ab6ebf2281 admin admin] PATH_INFO: `/v3/projects/dda6f6e14b004242a334de779124375e/users/8ac2f1cf8c8d460f940afcf61de9429b/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:42.357574 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbafff55-c642-4010-9494-35ab6ebf2281 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dda6f6e14b004242a334de779124375e, user_id=8ac2f1cf8c8d460f940afcf61de9429b, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:42.359178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-6ca24d62-9959-4e79-9c97-cdcbc7700671 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74eaf84e70f64c37803b49efcd669fb5', 'request_id': 'req-6ca24d62-9959-4e79-9c97-cdcbc7700671'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:42.360495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbafff55-c642-4010-9494-35ab6ebf2281 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:42.473287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-6ca24d62-9959-4e79-9c97-cdcbc7700671 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74eaf84e70f64c37803b49efcd669fb5', 'request_id': 'req-6ca24d62-9959-4e79-9c97-cdcbc7700671'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:42.481493 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-6ca24d62-9959-4e79-9c97-cdcbc7700671 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74eaf84e70f64c37803b49efcd669fb5', 'request_id': 'req-6ca24d62-9959-4e79-9c97-cdcbc7700671'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:42.492462 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-6ca24d62-9959-4e79-9c97-cdcbc7700671 admin admin] The token cache is being invalidate because project 74eaf84e70f64c37803b49efcd669fb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:45:42.492774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-6ca24d62-9959-4e79-9c97-cdcbc7700671 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:42.493451 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-6ca24d62-9959-4e79-9c97-cdcbc7700671 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:42.495097 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 903/1835] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:45:42 2022] DELETE /identity/v3/projects/74eaf84e70f64c37803b49efcd669fb5 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:42.495908 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca24d62-9959-4e79-9c97-cdcbc7700671 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:42.497248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-6ca24d62-9959-4e79-9c97-cdcbc7700671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:42.525958 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 933/1836] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:42 2022] PUT /identity/v3/projects/dda6f6e14b004242a334de779124375e/users/8ac2f1cf8c8d460f940afcf61de9429b/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:42.527501 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb75502-cb07-4e6e-8b39-46ad77e1a0bb None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:42.527782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb75502-cb07-4e6e-8b39-46ad77e1a0bb None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:42.528095 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb75502-cb07-4e6e-8b39-46ad77e1a0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:42.560000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-edb75502-cb07-4e6e-8b39-46ad77e1a0bb None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:42.561476 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-edb75502-cb07-4e6e-8b39-46ad77e1a0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:42.577020 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca24d62-9959-4e79-9c97-cdcbc7700671 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:42.631599 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 934/1837] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:42.633824 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d6941-985f-460d-9b97-68af16893980 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:42.634113 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d6941-985f-460d-9b97-68af16893980 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:42.634413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d6941-985f-460d-9b97-68af16893980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:42.643168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c1159-b23e-4d84-93e0-b4a50f9b342c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:42.643891 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c1159-b23e-4d84-93e0-b4a50f9b342c admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:42.644165 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c1159-b23e-4d84-93e0-b4a50f9b342c admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:42.644760 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c1159-b23e-4d84-93e0-b4a50f9b342c admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:42.645308 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c1159-b23e-4d84-93e0-b4a50f9b342c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:42.647306 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c1159-b23e-4d84-93e0-b4a50f9b342c admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:42.680382 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-e77d6941-985f-460d-9b97-68af16893980 None None] MFA Rules not processed for user `8ac2f1cf8c8d460f940afcf61de9429b`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:42.681760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-e77d6941-985f-460d-9b97-68af16893980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:42.709214 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:42.755890 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 904/1838] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:42 2022] POST /identity/v3/users => generated 284 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:42.757889 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa76b6-34b8-47ca-9729-283854bce363 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:42.758257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa76b6-34b8-47ca-9729-283854bce363 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:42.758553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa76b6-34b8-47ca-9729-283854bce363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:42.792173 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-43fa76b6-34b8-47ca-9729-283854bce363 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:42.793215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-43fa76b6-34b8-47ca-9729-283854bce363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:42.794939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 935/1839] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:42 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:42.795789 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d6941-985f-460d-9b97-68af16893980 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:42.808056 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 905/1840] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:42.808056 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa76b6-34b8-47ca-9729-283854bce363 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:42.808056 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-43fa76b6-34b8-47ca-9729-283854bce363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:42.813526 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d6941-985f-460d-9b97-68af16893980 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:42.818210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852bca0-c123-4cc5-b9fc-1d887fbf6d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:42.819230 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa76b6-34b8-47ca-9729-283854bce363 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:42.819354 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852bca0-c123-4cc5-b9fc-1d887fbf6d5a admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:42.819354 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852bca0-c123-4cc5-b9fc-1d887fbf6d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:42.820260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852bca0-c123-4cc5-b9fc-1d887fbf6d5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:42.821603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c852bca0-c123-4cc5-b9fc-1d887fbf6d5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:42.823394 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c852bca0-c123-4cc5-b9fc-1d887fbf6d5a admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:42.824199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a093f9-78ec-4db3-9ce6-ae795e17fdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:42.824867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a093f9-78ec-4db3-9ce6-ae795e17fdd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:42.825129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a093f9-78ec-4db3-9ce6-ae795e17fdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:42.825401 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a093f9-78ec-4db3-9ce6-ae795e17fdd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:42.831526 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a093f9-78ec-4db3-9ce6-ae795e17fdd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:42.833780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a093f9-78ec-4db3-9ce6-ae795e17fdd0 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:42.843800 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 936/1841] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:42.845352 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852bca0-c123-4cc5-b9fc-1d887fbf6d5a admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:42.860701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852bca0-c123-4cc5-b9fc-1d887fbf6d5a admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:42.877896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff37b8-a407-4a1a-b1ec-9be7ddaf390c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:42.878653 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff37b8-a407-4a1a-b1ec-9be7ddaf390c admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:42.879122 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff37b8-a407-4a1a-b1ec-9be7ddaf390c admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:42.879682 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff37b8-a407-4a1a-b1ec-9be7ddaf390c admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:42.880697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff37b8-a407-4a1a-b1ec-9be7ddaf390c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:45:42.881091 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff37b8-a407-4a1a-b1ec-9be7ddaf390c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:42.883238 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff37b8-a407-4a1a-b1ec-9be7ddaf390c admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:42.897494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 937/1842] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:45:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:45:42.898877 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff37b8-a407-4a1a-b1ec-9be7ddaf390c admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:42.909625 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff37b8-a407-4a1a-b1ec-9be7ddaf390c admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:42.940615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed624e1e-5647-4c25-9462-625e34548322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:42.941612 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed624e1e-5647-4c25-9462-625e34548322 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:42.941961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed624e1e-5647-4c25-9462-625e34548322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:42.942365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed624e1e-5647-4c25-9462-625e34548322 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:42.953537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed624e1e-5647-4c25-9462-625e34548322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:42.956419 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed624e1e-5647-4c25-9462-625e34548322 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:42.959600 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 906/1843] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:42 2022] POST /identity/v3/projects => generated 401 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:42.961320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a093f9-78ec-4db3-9ce6-ae795e17fdd0 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:42.985172 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 938/1844] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:42.985172 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed624e1e-5647-4c25-9462-625e34548322 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:43.001334 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed624e1e-5647-4c25-9462-625e34548322 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:43.091332 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a093f9-78ec-4db3-9ce6-ae795e17fdd0 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:43.103991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e18a0-bdee-41ca-ad53-3bf94035ddd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:43.104895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e18a0-bdee-41ca-ad53-3bf94035ddd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:43.105192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e18a0-bdee-41ca-ad53-3bf94035ddd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:43.105477 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e18a0-bdee-41ca-ad53-3bf94035ddd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:43.106190 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e18a0-bdee-41ca-ad53-3bf94035ddd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:43.108668 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e18a0-bdee-41ca-ad53-3bf94035ddd3 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:43.129883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a39594-ba04-4bd9-b505-7bf960351bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:43.129883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a39594-ba04-4bd9-b505-7bf960351bb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:43.130198 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a39594-ba04-4bd9-b505-7bf960351bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:43.130274 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a39594-ba04-4bd9-b505-7bf960351bb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:43.133439 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a39594-ba04-4bd9-b505-7bf960351bb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:43.135494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a39594-ba04-4bd9-b505-7bf960351bb2 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:43.200541 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 907/1845] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:42 2022] POST /identity/v3/projects => generated 369 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:43.201434 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e18a0-bdee-41ca-ad53-3bf94035ddd3 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:43.230753 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:43.236880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 939/1846] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:42 2022] POST /identity/v3/users => generated 307 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:43.236880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a39594-ba04-4bd9-b505-7bf960351bb2 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:43.254694 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a39594-ba04-4bd9-b505-7bf960351bb2 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:43.304426 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e18a0-bdee-41ca-ad53-3bf94035ddd3 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:43.313729 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f997d80-bc76-467b-9618-0924b2392940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:43.314944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f997d80-bc76-467b-9618-0924b2392940 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:43.315306 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f997d80-bc76-467b-9618-0924b2392940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:43.315639 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f997d80-bc76-467b-9618-0924b2392940 admin admin] PATH_INFO: `/v3/projects/bec2f9b1e8d048de96626c8ba3426eec/users/65d22c693dd7409ba095afffc1494081/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:43.334221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f997d80-bc76-467b-9618-0924b2392940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bec2f9b1e8d048de96626c8ba3426eec, user_id=65d22c693dd7409ba095afffc1494081, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:43.352292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f997d80-bc76-467b-9618-0924b2392940 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:43.395811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f6c71-b710-4c61-933c-32661486c04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:43.396666 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f6c71-b710-4c61-933c-32661486c04b admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:43.397613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f6c71-b710-4c61-933c-32661486c04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:43.399393 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f6c71-b710-4c61-933c-32661486c04b admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:43.400446 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f6c71-b710-4c61-933c-32661486c04b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:43.402961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f6c71-b710-4c61-933c-32661486c04b admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:43.519579 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 908/1847] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:43 2022] PUT /identity/v3/projects/bec2f9b1e8d048de96626c8ba3426eec/users/65d22c693dd7409ba095afffc1494081/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:43.521810 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f997d80-bc76-467b-9618-0924b2392940 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:43.531375 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 940/1848] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:43 2022] POST /identity/v3/users => generated 285 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:43.534400 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e0cdd-5061-4448-82e3-31be5dc8f407 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:43.535373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e0cdd-5061-4448-82e3-31be5dc8f407 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:43.535756 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e0cdd-5061-4448-82e3-31be5dc8f407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:43.582309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-bc9e0cdd-5061-4448-82e3-31be5dc8f407 None None] MFA Rules not processed for user `65d22c693dd7409ba095afffc1494081`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:43.584142 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-bc9e0cdd-5061-4448-82e3-31be5dc8f407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:43.603925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:43.637574 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f997d80-bc76-467b-9618-0924b2392940 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:43.644181 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1bbd7-6c8f-4e64-be28-615341923d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:43.645089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1bbd7-6c8f-4e64-be28-615341923d0c admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:43.645424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1bbd7-6c8f-4e64-be28-615341923d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:43.645802 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1bbd7-6c8f-4e64-be28-615341923d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:43.646864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa1bbd7-6c8f-4e64-be28-615341923d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:43.649481 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa1bbd7-6c8f-4e64-be28-615341923d0c admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:43.666442 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 909/1849] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:43.666442 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1bbd7-6c8f-4e64-be28-615341923d0c admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:43.686493 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1bbd7-6c8f-4e64-be28-615341923d0c admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:43.687880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 941/1850] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:43 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:45:43.688906 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e0cdd-5061-4448-82e3-31be5dc8f407 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:43.691574 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58bbf3-a1b5-408a-aa01-1f94b6b83bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:43.692239 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58bbf3-a1b5-408a-aa01-1f94b6b83bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:43.692549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58bbf3-a1b5-408a-aa01-1f94b6b83bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:43.692834 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58bbf3-a1b5-408a-aa01-1f94b6b83bc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:43.693558 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58bbf3-a1b5-408a-aa01-1f94b6b83bc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:43.695760 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58bbf3-a1b5-408a-aa01-1f94b6b83bc4 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:43.701314 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e0cdd-5061-4448-82e3-31be5dc8f407 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:43.707292 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f92285-f71a-42c6-8c13-be822dfebf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:43.708182 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f92285-f71a-42c6-8c13-be822dfebf7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:43.708492 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f92285-f71a-42c6-8c13-be822dfebf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:43.708914 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f92285-f71a-42c6-8c13-be822dfebf7d admin admin] PATH_INFO: `/v3/projects/d3e56568cfdf4c8abecbd794e857dc78/users/003cbb403a17404096e479a3d7deb9a5/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:43.713159 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 910/1851] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:43.714274 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58bbf3-a1b5-408a-aa01-1f94b6b83bc4 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:43.726294 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58bbf3-a1b5-408a-aa01-1f94b6b83bc4 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:43.734210 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8f506-177d-45c9-958f-64dcf2cde3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:43.734913 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8f506-177d-45c9-958f-64dcf2cde3f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:43.735185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8f506-177d-45c9-958f-64dcf2cde3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:43.740273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8f506-177d-45c9-958f-64dcf2cde3f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:43.740370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f92285-f71a-42c6-8c13-be822dfebf7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3e56568cfdf4c8abecbd794e857dc78, user_id=003cbb403a17404096e479a3d7deb9a5, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:43.740447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8f506-177d-45c9-958f-64dcf2cde3f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:43.740447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8f506-177d-45c9-958f-64dcf2cde3f1 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:43.740587 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f92285-f71a-42c6-8c13-be822dfebf7d admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:43.856931 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 942/1852] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:43 2022] PUT /identity/v3/projects/d3e56568cfdf4c8abecbd794e857dc78/users/003cbb403a17404096e479a3d7deb9a5/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:43.860669 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f92285-f71a-42c6-8c13-be822dfebf7d admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:43.878707 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 911/1853] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:43 2022] POST /identity/v3/projects => generated 365 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:43.878707 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8f506-177d-45c9-958f-64dcf2cde3f1 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:43.937557 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f92285-f71a-42c6-8c13-be822dfebf7d admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:43.942777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470ff81-81f0-4f8e-b87b-6003efb8539b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:43.943146 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470ff81-81f0-4f8e-b87b-6003efb8539b admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:43.943395 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470ff81-81f0-4f8e-b87b-6003efb8539b admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:43.943672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470ff81-81f0-4f8e-b87b-6003efb8539b admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:43.944483 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470ff81-81f0-4f8e-b87b-6003efb8539b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:43.946059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8f506-177d-45c9-958f-64dcf2cde3f1 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:43.946642 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470ff81-81f0-4f8e-b87b-6003efb8539b admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:43.951137 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6471a-a1c5-488c-a454-577afc44d5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:43.951885 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6471a-a1c5-488c-a454-577afc44d5bc admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:43.952167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6471a-a1c5-488c-a454-577afc44d5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:43.952472 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6471a-a1c5-488c-a454-577afc44d5bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:43.953197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6471a-a1c5-488c-a454-577afc44d5bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:43.954968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6471a-a1c5-488c-a454-577afc44d5bc admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:43.956524 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 943/1854] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:43.957306 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470ff81-81f0-4f8e-b87b-6003efb8539b admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:43.966528 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 912/1855] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:43.966528 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6471a-a1c5-488c-a454-577afc44d5bc admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:43.983319 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470ff81-81f0-4f8e-b87b-6003efb8539b admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:43.983319 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df70ee-ef72-49ed-a440-0e9852eaa9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:43.983319 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df70ee-ef72-49ed-a440-0e9852eaa9bb admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:43.983319 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df70ee-ef72-49ed-a440-0e9852eaa9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:43.983670 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df70ee-ef72-49ed-a440-0e9852eaa9bb admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:43.983670 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df70ee-ef72-49ed-a440-0e9852eaa9bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:43.983821 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6471a-a1c5-488c-a454-577afc44d5bc admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:43.983907 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df70ee-ef72-49ed-a440-0e9852eaa9bb admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:43.983991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8369374-b338-4364-b0a3-15d427d418c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:43.983991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8369374-b338-4364-b0a3-15d427d418c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:43.983991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8369374-b338-4364-b0a3-15d427d418c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:43.983991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8369374-b338-4364-b0a3-15d427d418c1 admin admin] PATH_INFO: `/v3/projects/5bc8701c27f94a1dbf29aa5b988e030e/users/686baef51fc54092a9b7d16b6d8474c8/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:44.000992 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8369374-b338-4364-b0a3-15d427d418c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bc8701c27f94a1dbf29aa5b988e030e, user_id=686baef51fc54092a9b7d16b6d8474c8, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:44.005007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8369374-b338-4364-b0a3-15d427d418c1 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:44.084358 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:44.188973 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 944/1856] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:43 2022] POST /identity/v3/users => generated 289 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:44.188973 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df70ee-ef72-49ed-a440-0e9852eaa9bb admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:44.199191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df70ee-ef72-49ed-a440-0e9852eaa9bb admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:44.204145 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5b908-df4e-4bda-8bc7-df805231f012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:44.205423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5b908-df4e-4bda-8bc7-df805231f012 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:44.205719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5b908-df4e-4bda-8bc7-df805231f012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:44.205999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5b908-df4e-4bda-8bc7-df805231f012 admin admin] PATH_INFO: `/v3/projects/d3e56568cfdf4c8abecbd794e857dc78/users/003cbb403a17404096e479a3d7deb9a5/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:44.208377 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5b908-df4e-4bda-8bc7-df805231f012 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3e56568cfdf4c8abecbd794e857dc78, user_id=003cbb403a17404096e479a3d7deb9a5, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:44.211452 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5b908-df4e-4bda-8bc7-df805231f012 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:44.216645 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:44.221560 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 913/1857] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:43 2022] PUT /identity/v3/projects/5bc8701c27f94a1dbf29aa5b988e030e/users/686baef51fc54092a9b7d16b6d8474c8/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:44.221560 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8369374-b338-4364-b0a3-15d427d418c1 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:44.250547 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 945/1858] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:44 2022] PUT /identity/v3/projects/d3e56568cfdf4c8abecbd794e857dc78/users/003cbb403a17404096e479a3d7deb9a5/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:44.250547 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890d36c-7580-4e11-a48d-c983bfe2a612 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:44.250547 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890d36c-7580-4e11-a48d-c983bfe2a612 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:44.250547 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890d36c-7580-4e11-a48d-c983bfe2a612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:44.321138 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-1890d36c-7580-4e11-a48d-c983bfe2a612 None None] MFA Rules not processed for user `686baef51fc54092a9b7d16b6d8474c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:44.325263 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-1890d36c-7580-4e11-a48d-c983bfe2a612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:44.351372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8369374-b338-4364-b0a3-15d427d418c1 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:44.459035 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699cb8f9-4a13-4730-9c4a-c9c974b85bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:44.459575 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cb8f9-4a13-4730-9c4a-c9c974b85bea admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:44.459958 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cb8f9-4a13-4730-9c4a-c9c974b85bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:44.460753 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cb8f9-4a13-4730-9c4a-c9c974b85bea admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:44.462220 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699cb8f9-4a13-4730-9c4a-c9c974b85bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:44.464013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699cb8f9-4a13-4730-9c4a-c9c974b85bea admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:44.475252 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 914/1859] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:44.476913 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db970a8d-3cab-4ce3-b3a9-eb5b92a15281 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:44.477206 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db970a8d-3cab-4ce3-b3a9-eb5b92a15281 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:44.477483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db970a8d-3cab-4ce3-b3a9-eb5b92a15281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:44.502653 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 946/1860] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:44 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:44.510663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890d36c-7580-4e11-a48d-c983bfe2a612 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:44.514398 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-db970a8d-3cab-4ce3-b3a9-eb5b92a15281 None None] MFA Rules not processed for user `003cbb403a17404096e479a3d7deb9a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:44.514398 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-db970a8d-3cab-4ce3-b3a9-eb5b92a15281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:44.522643 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890d36c-7580-4e11-a48d-c983bfe2a612 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:44.532886 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe17113-daf5-4f73-b8ec-df5c6b024a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:44.537305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe17113-daf5-4f73-b8ec-df5c6b024a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:44.537305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe17113-daf5-4f73-b8ec-df5c6b024a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:44.537305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe17113-daf5-4f73-b8ec-df5c6b024a0b admin admin] PATH_INFO: `/v3/projects/31bc81f62bd343bbb24ded4412e8206e/users/97b06217e56142ceaf04da1c772f33a4/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:44.563193 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe17113-daf5-4f73-b8ec-df5c6b024a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=31bc81f62bd343bbb24ded4412e8206e, user_id=97b06217e56142ceaf04da1c772f33a4, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:44.571877 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe17113-daf5-4f73-b8ec-df5c6b024a0b admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:44.601716 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 915/1861] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:44 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:44.603748 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db970a8d-3cab-4ce3-b3a9-eb5b92a15281 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:44.606067 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-db970a8d-3cab-4ce3-b3a9-eb5b92a15281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:44.634716 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:44.672116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 947/1862] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:44 2022] PUT /identity/v3/projects/31bc81f62bd343bbb24ded4412e8206e/users/97b06217e56142ceaf04da1c772f33a4/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:44.677080 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe17113-daf5-4f73-b8ec-df5c6b024a0b admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:44.691380 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db970a8d-3cab-4ce3-b3a9-eb5b92a15281 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:44.697401 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:44.779303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f62d19-a5bc-4214-bbff-71a6beb2721a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e75e5ae3ca45d0b503f6e52f399558', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:44.781299 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f62d19-a5bc-4214-bbff-71a6beb2721a service glance] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:44.781586 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f62d19-a5bc-4214-bbff-71a6beb2721a service glance] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:44.781918 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f62d19-a5bc-4214-bbff-71a6beb2721a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:44.793512 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe17113-daf5-4f73-b8ec-df5c6b024a0b admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:44.800018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4492c73e-533e-4eb1-b6c7-30101906a45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:44.800717 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492c73e-533e-4eb1-b6c7-30101906a45d admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:44.801003 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492c73e-533e-4eb1-b6c7-30101906a45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:44.801279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492c73e-533e-4eb1-b6c7-30101906a45d admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:44.802037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4492c73e-533e-4eb1-b6c7-30101906a45d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:44.804406 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4492c73e-533e-4eb1-b6c7-30101906a45d admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:44.817919 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 948/1863] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:45:44.819382 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4492c73e-533e-4eb1-b6c7-30101906a45d admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:44.824733 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-4492c73e-533e-4eb1-b6c7-30101906a45d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:44.872841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f62d19-a5bc-4214-bbff-71a6beb2721a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:44.874726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f62d19-a5bc-4214-bbff-71a6beb2721a service glance] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:44.887252 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 916/1864] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:44 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:44.896356 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f62d19-a5bc-4214-bbff-71a6beb2721a service glance] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:44.908222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4492c73e-533e-4eb1-b6c7-30101906a45d admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:44.912839 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b8787-0739-477c-aae9-1b3c17e06846 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:44.913629 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f62d19-a5bc-4214-bbff-71a6beb2721a service glance] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:44.914559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b8787-0739-477c-aae9-1b3c17e06846 service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:44.914960 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b8787-0739-477c-aae9-1b3c17e06846 service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:44.915240 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b8787-0739-477c-aae9-1b3c17e06846 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:44.919941 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c7fce-c742-472b-805e-c84809b044af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:44.921372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c7fce-c742-472b-805e-c84809b044af admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:44.921818 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c7fce-c742-472b-805e-c84809b044af admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:44.922264 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c7fce-c742-472b-805e-c84809b044af admin admin] PATH_INFO: `/v3/projects/31bc81f62bd343bbb24ded4412e8206e/users/97b06217e56142ceaf04da1c772f33a4/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:44.924379 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c7fce-c742-472b-805e-c84809b044af admin admin] RBAC: Authorizing `identity:create_grant(project_id=31bc81f62bd343bbb24ded4412e8206e, user_id=97b06217e56142ceaf04da1c772f33a4, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:44.927095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c7fce-c742-472b-805e-c84809b044af admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:44.940919 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 917/1865] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:44 2022] PUT /identity/v3/projects/31bc81f62bd343bbb24ded4412e8206e/users/97b06217e56142ceaf04da1c772f33a4/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:44.952516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24442d7-bedd-4e05-90b2-22c37b0103e7 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:44.957470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24442d7-bedd-4e05-90b2-22c37b0103e7 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:44.957470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24442d7-bedd-4e05-90b2-22c37b0103e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:44.999499 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-d24442d7-bedd-4e05-90b2-22c37b0103e7 None None] MFA Rules not processed for user `97b06217e56142ceaf04da1c772f33a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:45.002271 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-d24442d7-bedd-4e05-90b2-22c37b0103e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:45.017833 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b8787-0739-477c-aae9-1b3c17e06846 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:45.020839 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b8787-0739-477c-aae9-1b3c17e06846 service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:45.044855 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 949/1866] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:44 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:45.121015 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 918/1867] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:45.180349 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:45.208733 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b8787-0739-477c-aae9-1b3c17e06846 service cinder] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:45.223929 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b8787-0739-477c-aae9-1b3c17e06846 service cinder] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:45.270605 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:45.330541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c1c7b-e6e0-4cd0-bbde-90e4764f71c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:45.331336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c1c7b-e6e0-4cd0-bbde-90e4764f71c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:45.331686 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c1c7b-e6e0-4cd0-bbde-90e4764f71c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:45.331891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c1c7b-e6e0-4cd0-bbde-90e4764f71c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:45.342442 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c1c7b-e6e0-4cd0-bbde-90e4764f71c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:45.343747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c1c7b-e6e0-4cd0-bbde-90e4764f71c6 service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:45.358526 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 950/1868] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:45 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:45.661757 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:46.167251 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24442d7-bedd-4e05-90b2-22c37b0103e7 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:46.168685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-d24442d7-bedd-4e05-90b2-22c37b0103e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:46.181813 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24442d7-bedd-4e05-90b2-22c37b0103e7 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:46.185968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5138b-8806-4a73-b31b-caef23ec8617 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:46.186209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5138b-8806-4a73-b31b-caef23ec8617 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:46.186528 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5138b-8806-4a73-b31b-caef23ec8617 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:46.186801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5138b-8806-4a73-b31b-caef23ec8617 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:46.250709 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c5138b-8806-4a73-b31b-caef23ec8617 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:46.252356 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c5138b-8806-4a73-b31b-caef23ec8617 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:46.263025 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 919/1869] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:46 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:47.847715 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c1c7b-e6e0-4cd0-bbde-90e4764f71c6 service cinder] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:47.999648 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c1c7b-e6e0-4cd0-bbde-90e4764f71c6 service cinder] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:48.005441 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e91d84-df72-4f87-bd2b-7b0d75454687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:48.006742 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e91d84-df72-4f87-bd2b-7b0d75454687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:48.007338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e91d84-df72-4f87-bd2b-7b0d75454687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:48.007898 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e91d84-df72-4f87-bd2b-7b0d75454687 admin admin] PATH_INFO: `/v3/users/8ac2f1cf8c8d460f940afcf61de9429b` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:48.010633 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e91d84-df72-4f87-bd2b-7b0d75454687 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ac2f1cf8c8d460f940afcf61de9429b)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:48.012755 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e91d84-df72-4f87-bd2b-7b0d75454687 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:48.246357 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-a4e91d84-df72-4f87-bd2b-7b0d75454687 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ac2f1cf8c8d460f940afcf61de9429b'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:48.264980 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-a4e91d84-df72-4f87-bd2b-7b0d75454687 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ac2f1cf8c8d460f940afcf61de9429b'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:48.270868 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-a4e91d84-df72-4f87-bd2b-7b0d75454687 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ac2f1cf8c8d460f940afcf61de9429b'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:48.271994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-a4e91d84-df72-4f87-bd2b-7b0d75454687 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ac2f1cf8c8d460f940afcf61de9429b'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:48.293507 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-a4e91d84-df72-4f87-bd2b-7b0d75454687 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ac2f1cf8c8d460f940afcf61de9429b'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:48.405684 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 951/1870] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:45:47 2022] DELETE /identity/v3/users/8ac2f1cf8c8d460f940afcf61de9429b => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:48.410605 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce541658-0e24-4aca-a76c-cd893b02e50d None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:48.411496 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce541658-0e24-4aca-a76c-cd893b02e50d None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:48.411979 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce541658-0e24-4aca-a76c-cd893b02e50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:48.466008 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-ce541658-0e24-4aca-a76c-cd893b02e50d None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:48.466008 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-ce541658-0e24-4aca-a76c-cd893b02e50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:48.593235 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 920/1871] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:48.624215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e91d84-df72-4f87-bd2b-7b0d75454687 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:48.625884 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-a4e91d84-df72-4f87-bd2b-7b0d75454687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:48.705744 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:48.735021 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e91d84-df72-4f87-bd2b-7b0d75454687 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:48.740650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700b8f7-90fc-4aa6-a170-e5eb81ef8223 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:48.746988 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700b8f7-90fc-4aa6-a170-e5eb81ef8223 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:48.746988 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700b8f7-90fc-4aa6-a170-e5eb81ef8223 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:48.746988 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700b8f7-90fc-4aa6-a170-e5eb81ef8223 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:48.758907 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700b8f7-90fc-4aa6-a170-e5eb81ef8223 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:48.760546 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700b8f7-90fc-4aa6-a170-e5eb81ef8223 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:48.775499 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 952/1872] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:50.049276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce541658-0e24-4aca-a76c-cd893b02e50d None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:50.055309 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-ce541658-0e24-4aca-a76c-cd893b02e50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:50.065129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce541658-0e24-4aca-a76c-cd893b02e50d None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:50.073794 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637fbbc-f24f-4aad-a171-10d7375eda58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:50.074721 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637fbbc-f24f-4aad-a171-10d7375eda58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:50.077151 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637fbbc-f24f-4aad-a171-10d7375eda58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:50.077151 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637fbbc-f24f-4aad-a171-10d7375eda58 admin admin] PATH_INFO: `/v3/projects/dda6f6e14b004242a334de779124375e` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:50.077151 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637fbbc-f24f-4aad-a171-10d7375eda58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dda6f6e14b004242a334de779124375e)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:50.078676 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637fbbc-f24f-4aad-a171-10d7375eda58 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:50.188378 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c637fbbc-f24f-4aad-a171-10d7375eda58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dda6f6e14b004242a334de779124375e', 'request_id': 'req-c637fbbc-f24f-4aad-a171-10d7375eda58'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:50.193003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:50.256344 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c637fbbc-f24f-4aad-a171-10d7375eda58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dda6f6e14b004242a334de779124375e', 'request_id': 'req-c637fbbc-f24f-4aad-a171-10d7375eda58'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:50.265470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c637fbbc-f24f-4aad-a171-10d7375eda58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dda6f6e14b004242a334de779124375e', 'request_id': 'req-c637fbbc-f24f-4aad-a171-10d7375eda58'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:50.279424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c637fbbc-f24f-4aad-a171-10d7375eda58 admin admin] The token cache is being invalidate because project dda6f6e14b004242a334de779124375e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:45:50.280017 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c637fbbc-f24f-4aad-a171-10d7375eda58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:50.280726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c637fbbc-f24f-4aad-a171-10d7375eda58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:50.282303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 921/1873] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:45:50 2022] DELETE /identity/v3/projects/dda6f6e14b004242a334de779124375e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:50.286588 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700b8f7-90fc-4aa6-a170-e5eb81ef8223 service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:50.299183 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700b8f7-90fc-4aa6-a170-e5eb81ef8223 service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:50.413374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8856139a-d209-41e8-99e1-be113f2546f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:50.415020 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856139a-d209-41e8-99e1-be113f2546f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:50.415020 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856139a-d209-41e8-99e1-be113f2546f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:50.415020 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856139a-d209-41e8-99e1-be113f2546f7 admin admin] PATH_INFO: `/v3/users/003cbb403a17404096e479a3d7deb9a5` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:50.416294 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8856139a-d209-41e8-99e1-be113f2546f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=003cbb403a17404096e479a3d7deb9a5)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:50.418503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8856139a-d209-41e8-99e1-be113f2546f7 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:50.668215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-8856139a-d209-41e8-99e1-be113f2546f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '003cbb403a17404096e479a3d7deb9a5'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:50.676882 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-8856139a-d209-41e8-99e1-be113f2546f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '003cbb403a17404096e479a3d7deb9a5'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:50.677588 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-8856139a-d209-41e8-99e1-be113f2546f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '003cbb403a17404096e479a3d7deb9a5'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:50.678187 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-8856139a-d209-41e8-99e1-be113f2546f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '003cbb403a17404096e479a3d7deb9a5'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:50.691652 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-8856139a-d209-41e8-99e1-be113f2546f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '003cbb403a17404096e479a3d7deb9a5'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:50.787453 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 953/1874] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:45:50 2022] DELETE /identity/v3/users/003cbb403a17404096e479a3d7deb9a5 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:45:51.199301 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637fbbc-f24f-4aad-a171-10d7375eda58 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:51.201377 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-c637fbbc-f24f-4aad-a171-10d7375eda58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:51.220176 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637fbbc-f24f-4aad-a171-10d7375eda58 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:51.283682 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18314010-f358-40d5-8776-47239954cbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:51.284511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18314010-f358-40d5-8776-47239954cbb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:51.284797 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18314010-f358-40d5-8776-47239954cbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:51.285103 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18314010-f358-40d5-8776-47239954cbb5 admin admin] PATH_INFO: `/v3/projects/d3e56568cfdf4c8abecbd794e857dc78` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:51.286465 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18314010-f358-40d5-8776-47239954cbb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3e56568cfdf4c8abecbd794e857dc78)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:51.289039 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18314010-f358-40d5-8776-47239954cbb5 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:51.434202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-18314010-f358-40d5-8776-47239954cbb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3e56568cfdf4c8abecbd794e857dc78', 'request_id': 'req-18314010-f358-40d5-8776-47239954cbb5'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:51.535196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-18314010-f358-40d5-8776-47239954cbb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3e56568cfdf4c8abecbd794e857dc78', 'request_id': 'req-18314010-f358-40d5-8776-47239954cbb5'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:51.546452 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-18314010-f358-40d5-8776-47239954cbb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3e56568cfdf4c8abecbd794e857dc78', 'request_id': 'req-18314010-f358-40d5-8776-47239954cbb5'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:51.562324 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-18314010-f358-40d5-8776-47239954cbb5 admin admin] The token cache is being invalidate because project d3e56568cfdf4c8abecbd794e857dc78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:45:51.562324 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-18314010-f358-40d5-8776-47239954cbb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:51.562872 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-18314010-f358-40d5-8776-47239954cbb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:51.565205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 922/1875] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:45:51 2022] DELETE /identity/v3/projects/d3e56568cfdf4c8abecbd794e857dc78 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:51.570018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71b78c-8e7a-44d4-8794-19cc97d4a2a8 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:51.570562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71b78c-8e7a-44d4-8794-19cc97d4a2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:51.570648 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71b78c-8e7a-44d4-8794-19cc97d4a2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:51.593899 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-4d71b78c-8e7a-44d4-8794-19cc97d4a2a8 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:51.595227 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-4d71b78c-8e7a-44d4-8794-19cc97d4a2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:51.665116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 954/1876] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:51.673479 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4dd86-039d-4110-b881-836962ea7589 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:51.673765 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4dd86-039d-4110-b881-836962ea7589 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:51.674043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4dd86-039d-4110-b881-836962ea7589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:51.695575 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-3da4dd86-039d-4110-b881-836962ea7589 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:51.696708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-3da4dd86-039d-4110-b881-836962ea7589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:51.705487 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 923/1877] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:51.711260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71b78c-8e7a-44d4-8794-19cc97d4a2a8 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:51.721038 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71b78c-8e7a-44d4-8794-19cc97d4a2a8 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:51.729137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d57f00-78e4-45b1-beb9-669694965e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:51.729954 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d57f00-78e4-45b1-beb9-669694965e0b admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:51.730348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d57f00-78e4-45b1-beb9-669694965e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:51.730722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d57f00-78e4-45b1-beb9-669694965e0b admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:51.731762 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d57f00-78e4-45b1-beb9-669694965e0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:45:51.732815 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d57f00-78e4-45b1-beb9-669694965e0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:51.735088 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d57f00-78e4-45b1-beb9-669694965e0b admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:51.737481 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:51.767251 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 955/1878] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:45:51.772019 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4dd86-039d-4110-b881-836962ea7589 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:51.784442 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4dd86-039d-4110-b881-836962ea7589 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:51.789719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319bfcb8-c834-4078-a042-b3b3e5a75c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:51.791037 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bfcb8-c834-4078-a042-b3b3e5a75c21 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:51.791411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bfcb8-c834-4078-a042-b3b3e5a75c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:51.791751 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bfcb8-c834-4078-a042-b3b3e5a75c21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:51.792534 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319bfcb8-c834-4078-a042-b3b3e5a75c21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:51.794525 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319bfcb8-c834-4078-a042-b3b3e5a75c21 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:51.900000 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 924/1879] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:51 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:51.901660 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d57f00-78e4-45b1-beb9-669694965e0b admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:51.915019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d57f00-78e4-45b1-beb9-669694965e0b admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:52.000514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d5e34-d801-4d6a-8d19-433adaf94c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:52.001533 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d5e34-d801-4d6a-8d19-433adaf94c75 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:52.001990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d5e34-d801-4d6a-8d19-433adaf94c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:52.002980 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d5e34-d801-4d6a-8d19-433adaf94c75 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:52.002980 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d5e34-d801-4d6a-8d19-433adaf94c75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:52.004934 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d5e34-d801-4d6a-8d19-433adaf94c75 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:52.129371 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 956/1880] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:51 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:52.135092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319bfcb8-c834-4078-a042-b3b3e5a75c21 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:52.145368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319bfcb8-c834-4078-a042-b3b3e5a75c21 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:52.150116 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf9411-0c06-438c-bad1-fe435eba9bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:52.150872 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf9411-0c06-438c-bad1-fe435eba9bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:52.151160 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf9411-0c06-438c-bad1-fe435eba9bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:52.151438 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf9411-0c06-438c-bad1-fe435eba9bf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:52.152166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf9411-0c06-438c-bad1-fe435eba9bf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:52.153755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf9411-0c06-438c-bad1-fe435eba9bf1 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:52.164194 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 925/1881] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:52.168268 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d5e34-d801-4d6a-8d19-433adaf94c75 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:52.180520 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d5e34-d801-4d6a-8d19-433adaf94c75 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:52.184790 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5dcbc-0d2c-4597-89e7-d16926a36ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:52.185451 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5dcbc-0d2c-4597-89e7-d16926a36ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:52.185734 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5dcbc-0d2c-4597-89e7-d16926a36ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:52.186010 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5dcbc-0d2c-4597-89e7-d16926a36ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:52.186781 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c5dcbc-0d2c-4597-89e7-d16926a36ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:52.188379 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c5dcbc-0d2c-4597-89e7-d16926a36ce1 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:52.199273 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 957/1882] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:52.203746 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf9411-0c06-438c-bad1-fe435eba9bf1 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:52.213914 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf9411-0c06-438c-bad1-fe435eba9bf1 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:52.219642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0acbc-3c3a-4291-818a-3fcb00d47dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:52.221423 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0acbc-3c3a-4291-818a-3fcb00d47dbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:52.221423 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0acbc-3c3a-4291-818a-3fcb00d47dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:52.221423 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0acbc-3c3a-4291-818a-3fcb00d47dbe admin admin] PATH_INFO: `/v3/projects/1f4fb08762e84c9fa36a6c0841ad8cb8/users/3793528e80e84c9a93f1294f2d2b9ad4/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:52.238682 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0acbc-3c3a-4291-818a-3fcb00d47dbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f4fb08762e84c9fa36a6c0841ad8cb8, user_id=3793528e80e84c9a93f1294f2d2b9ad4, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:52.242789 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0acbc-3c3a-4291-818a-3fcb00d47dbe admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:52.323537 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:52.328434 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 926/1883] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:52 2022] PUT /identity/v3/projects/1f4fb08762e84c9fa36a6c0841ad8cb8/users/3793528e80e84c9a93f1294f2d2b9ad4/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:52.339156 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446340f-7369-477a-897f-279c46571039 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:52.339446 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446340f-7369-477a-897f-279c46571039 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:52.339751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446340f-7369-477a-897f-279c46571039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:52.367133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-1446340f-7369-477a-897f-279c46571039 None None] MFA Rules not processed for user `3793528e80e84c9a93f1294f2d2b9ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:52.370665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-1446340f-7369-477a-897f-279c46571039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:52.435501 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 958/1884] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:52.440056 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0acbc-3c3a-4291-818a-3fcb00d47dbe admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:52.509135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0acbc-3c3a-4291-818a-3fcb00d47dbe admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:52.514758 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a35130-e852-4353-9d87-d7bfbfdf91b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:52.516496 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a35130-e852-4353-9d87-d7bfbfdf91b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:52.516496 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a35130-e852-4353-9d87-d7bfbfdf91b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:52.516496 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a35130-e852-4353-9d87-d7bfbfdf91b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:52.516741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a35130-e852-4353-9d87-d7bfbfdf91b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:52.518655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a35130-e852-4353-9d87-d7bfbfdf91b0 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:52.628067 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 927/1885] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:52 2022] POST /identity/v3/projects => generated 387 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:45:52.633557 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446340f-7369-477a-897f-279c46571039 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:52.644053 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446340f-7369-477a-897f-279c46571039 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:52.700560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd7aba-1477-4aff-b8bb-f18f88954071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:52.701230 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd7aba-1477-4aff-b8bb-f18f88954071 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:52.701511 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd7aba-1477-4aff-b8bb-f18f88954071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:52.701781 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd7aba-1477-4aff-b8bb-f18f88954071 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:52.702544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd7aba-1477-4aff-b8bb-f18f88954071 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:52.704474 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd7aba-1477-4aff-b8bb-f18f88954071 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:52.712241 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:52.796265 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 959/1886] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:52 2022] POST /identity/v3/users => generated 300 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:52.797605 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a35130-e852-4353-9d87-d7bfbfdf91b0 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:52.808990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a35130-e852-4353-9d87-d7bfbfdf91b0 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:52.815470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b1a9e-1726-4564-8e7d-18bc9f574d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:52.816140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b1a9e-1726-4564-8e7d-18bc9f574d4a admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:52.816423 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b1a9e-1726-4564-8e7d-18bc9f574d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:52.817027 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b1a9e-1726-4564-8e7d-18bc9f574d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:52.817772 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b1a9e-1726-4564-8e7d-18bc9f574d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:52.828966 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b1a9e-1726-4564-8e7d-18bc9f574d4a admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:52.849487 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 928/1887] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:52.853874 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd7aba-1477-4aff-b8bb-f18f88954071 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:52.864610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd7aba-1477-4aff-b8bb-f18f88954071 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:52.875511 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6664c9f-d3a2-404a-8647-8ca6c0c97134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:52.875511 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6664c9f-d3a2-404a-8647-8ca6c0c97134 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:52.875511 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6664c9f-d3a2-404a-8647-8ca6c0c97134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:52.875836 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6664c9f-d3a2-404a-8647-8ca6c0c97134 admin admin] PATH_INFO: `/v3/projects/1d4ab182ab5d45b3bf3bb4cfc85e2215/users/0c2a9c3c96da4559b3ac10ec93646784/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:52.895371 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6664c9f-d3a2-404a-8647-8ca6c0c97134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d4ab182ab5d45b3bf3bb4cfc85e2215, user_id=0c2a9c3c96da4559b3ac10ec93646784, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:52.899066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6664c9f-d3a2-404a-8647-8ca6c0c97134 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:52.993241 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 960/1888] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:52 2022] PUT /identity/v3/projects/1d4ab182ab5d45b3bf3bb4cfc85e2215/users/0c2a9c3c96da4559b3ac10ec93646784/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:52.997694 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b1a9e-1726-4564-8e7d-18bc9f574d4a admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:53.008689 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b1a9e-1726-4564-8e7d-18bc9f574d4a admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:53.086501 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c2436-1b57-45e5-8b12-d1149a60ae61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:53.087625 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c2436-1b57-45e5-8b12-d1149a60ae61 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:53.088062 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c2436-1b57-45e5-8b12-d1149a60ae61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:53.088418 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c2436-1b57-45e5-8b12-d1149a60ae61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:53.089469 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c2436-1b57-45e5-8b12-d1149a60ae61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:53.091316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c2436-1b57-45e5-8b12-d1149a60ae61 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:53.105435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 929/1889] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:45:53.110110 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6664c9f-d3a2-404a-8647-8ca6c0c97134 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:53.128428 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6664c9f-d3a2-404a-8647-8ca6c0c97134 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:53.135430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548dd6c-1993-437e-8f08-0c8fcec0757f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:53.136332 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548dd6c-1993-437e-8f08-0c8fcec0757f admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:53.136614 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548dd6c-1993-437e-8f08-0c8fcec0757f admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:53.136889 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548dd6c-1993-437e-8f08-0c8fcec0757f admin admin] PATH_INFO: `/v3/projects/1d4ab182ab5d45b3bf3bb4cfc85e2215/users/0c2a9c3c96da4559b3ac10ec93646784/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:53.139312 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a548dd6c-1993-437e-8f08-0c8fcec0757f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d4ab182ab5d45b3bf3bb4cfc85e2215, user_id=0c2a9c3c96da4559b3ac10ec93646784, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:53.141990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a548dd6c-1993-437e-8f08-0c8fcec0757f admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:53.153339 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 961/1890] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:53 2022] PUT /identity/v3/projects/1d4ab182ab5d45b3bf3bb4cfc85e2215/users/0c2a9c3c96da4559b3ac10ec93646784/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:53.164269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291bbaa-6600-4e66-8263-3d3f67c73961 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:53.164553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291bbaa-6600-4e66-8263-3d3f67c73961 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:53.164829 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291bbaa-6600-4e66-8263-3d3f67c73961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:53.191758 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-3291bbaa-6600-4e66-8263-3d3f67c73961 None None] MFA Rules not processed for user `0c2a9c3c96da4559b3ac10ec93646784`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:53.192970 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-3291bbaa-6600-4e66-8263-3d3f67c73961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:53.264011 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:53.271869 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 930/1891] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:53.284052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548dd6c-1993-437e-8f08-0c8fcec0757f admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:53.286029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-a548dd6c-1993-437e-8f08-0c8fcec0757f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:53.343851 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548dd6c-1993-437e-8f08-0c8fcec0757f admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:53.347545 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a415bc-b8f9-4547-9235-5157c90a5bf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:53.348184 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a415bc-b8f9-4547-9235-5157c90a5bf4 service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:53.348472 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a415bc-b8f9-4547-9235-5157c90a5bf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:53.348746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a415bc-b8f9-4547-9235-5157c90a5bf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:53.357931 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a415bc-b8f9-4547-9235-5157c90a5bf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:53.359402 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a415bc-b8f9-4547-9235-5157c90a5bf4 service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:53.369582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 962/1892] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:53.397544 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291bbaa-6600-4e66-8263-3d3f67c73961 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:53.457963 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291bbaa-6600-4e66-8263-3d3f67c73961 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:53.462884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf209e-5607-421d-9a52-b8550d08838a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:53.463739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf209e-5607-421d-9a52-b8550d08838a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:53.464021 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf209e-5607-421d-9a52-b8550d08838a admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:53.464298 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf209e-5607-421d-9a52-b8550d08838a admin admin] PATH_INFO: `/v3/users/3793528e80e84c9a93f1294f2d2b9ad4` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:53.465546 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf209e-5607-421d-9a52-b8550d08838a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3793528e80e84c9a93f1294f2d2b9ad4)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:53.467505 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf209e-5607-421d-9a52-b8550d08838a admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:53.622004 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-debf209e-5607-421d-9a52-b8550d08838a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3793528e80e84c9a93f1294f2d2b9ad4'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:53.628863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-debf209e-5607-421d-9a52-b8550d08838a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3793528e80e84c9a93f1294f2d2b9ad4'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:53.629538 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-debf209e-5607-421d-9a52-b8550d08838a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3793528e80e84c9a93f1294f2d2b9ad4'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:53.641177 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-debf209e-5607-421d-9a52-b8550d08838a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3793528e80e84c9a93f1294f2d2b9ad4'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:53.694035 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-debf209e-5607-421d-9a52-b8550d08838a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3793528e80e84c9a93f1294f2d2b9ad4'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:53.700149 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 931/1893] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:45:53 2022] DELETE /identity/v3/users/3793528e80e84c9a93f1294f2d2b9ad4 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:53.704751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05805847-50fc-4186-be0d-c92810b0f61a None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:53.705199 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05805847-50fc-4186-be0d-c92810b0f61a None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:53.705285 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05805847-50fc-4186-be0d-c92810b0f61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:53.723083 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-05805847-50fc-4186-be0d-c92810b0f61a None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:53.724278 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-05805847-50fc-4186-be0d-c92810b0f61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:53.785006 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 963/1894] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:53.795074 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf209e-5607-421d-9a52-b8550d08838a admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:53.796491 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-debf209e-5607-421d-9a52-b8550d08838a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:53.853146 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf209e-5607-421d-9a52-b8550d08838a admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:53.856920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebc68d-48df-418d-862f-c7ff684f2bb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:53.857938 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebc68d-48df-418d-862f-c7ff684f2bb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:53.858224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebc68d-48df-418d-862f-c7ff684f2bb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:53.858529 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebc68d-48df-418d-862f-c7ff684f2bb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:53.868606 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ebc68d-48df-418d-862f-c7ff684f2bb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:53.870024 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ebc68d-48df-418d-862f-c7ff684f2bb4 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:53.879429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 932/1895] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:45:54.475572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05805847-50fc-4186-be0d-c92810b0f61a None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:54.476985 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-05805847-50fc-4186-be0d-c92810b0f61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:54.489250 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05805847-50fc-4186-be0d-c92810b0f61a None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:54.494709 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd109d1-c935-43eb-a030-993a863b6f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:54.495744 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd109d1-c935-43eb-a030-993a863b6f96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:54.496025 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd109d1-c935-43eb-a030-993a863b6f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:54.496301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd109d1-c935-43eb-a030-993a863b6f96 admin admin] PATH_INFO: `/v3/projects/1f4fb08762e84c9fa36a6c0841ad8cb8` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:54.497582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd109d1-c935-43eb-a030-993a863b6f96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f4fb08762e84c9fa36a6c0841ad8cb8)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:54.499612 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd109d1-c935-43eb-a030-993a863b6f96 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:54.615136 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-acd109d1-c935-43eb-a030-993a863b6f96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f4fb08762e84c9fa36a6c0841ad8cb8', 'request_id': 'req-acd109d1-c935-43eb-a030-993a863b6f96'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:54.619556 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:54.699170 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-acd109d1-c935-43eb-a030-993a863b6f96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f4fb08762e84c9fa36a6c0841ad8cb8', 'request_id': 'req-acd109d1-c935-43eb-a030-993a863b6f96'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:54.706301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-acd109d1-c935-43eb-a030-993a863b6f96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f4fb08762e84c9fa36a6c0841ad8cb8', 'request_id': 'req-acd109d1-c935-43eb-a030-993a863b6f96'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:54.717944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-acd109d1-c935-43eb-a030-993a863b6f96 admin admin] The token cache is being invalidate because project 1f4fb08762e84c9fa36a6c0841ad8cb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:45:54.718240 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-acd109d1-c935-43eb-a030-993a863b6f96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:54.718916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-acd109d1-c935-43eb-a030-993a863b6f96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:54.720463 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 964/1896] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:45:54 2022] DELETE /identity/v3/projects/1f4fb08762e84c9fa36a6c0841ad8cb8 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:54.724524 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebc68d-48df-418d-862f-c7ff684f2bb4 service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:54.735639 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebc68d-48df-418d-862f-c7ff684f2bb4 service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:54.749467 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:54.793623 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca99457-673d-47d8-b857-f099aae52260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:54.794551 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca99457-673d-47d8-b857-f099aae52260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:54.794829 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca99457-673d-47d8-b857-f099aae52260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:54.795137 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca99457-673d-47d8-b857-f099aae52260 admin admin] PATH_INFO: `/v3/users/0c2a9c3c96da4559b3ac10ec93646784` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:54.796311 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca99457-673d-47d8-b857-f099aae52260 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c2a9c3c96da4559b3ac10ec93646784)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:54.798253 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca99457-673d-47d8-b857-f099aae52260 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:55.048741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-dca99457-673d-47d8-b857-f099aae52260 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c2a9c3c96da4559b3ac10ec93646784'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:55.057874 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-dca99457-673d-47d8-b857-f099aae52260 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c2a9c3c96da4559b3ac10ec93646784'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:55.059054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-dca99457-673d-47d8-b857-f099aae52260 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c2a9c3c96da4559b3ac10ec93646784'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:55.074811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-dca99457-673d-47d8-b857-f099aae52260 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c2a9c3c96da4559b3ac10ec93646784'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:55.152732 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-dca99457-673d-47d8-b857-f099aae52260 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c2a9c3c96da4559b3ac10ec93646784'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:55.169252 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 933/1897] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:45:54 2022] DELETE /identity/v3/users/0c2a9c3c96da4559b3ac10ec93646784 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:55.719251 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd109d1-c935-43eb-a030-993a863b6f96 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:55.720699 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-acd109d1-c935-43eb-a030-993a863b6f96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:55.746739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd109d1-c935-43eb-a030-993a863b6f96 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:55.827419 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba286541-8bc4-44e3-a243-d4556fd21025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:55.829588 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba286541-8bc4-44e3-a243-d4556fd21025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:55.830291 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba286541-8bc4-44e3-a243-d4556fd21025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:55.830757 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba286541-8bc4-44e3-a243-d4556fd21025 admin admin] PATH_INFO: `/v3/projects/1d4ab182ab5d45b3bf3bb4cfc85e2215` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:55.832251 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba286541-8bc4-44e3-a243-d4556fd21025 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d4ab182ab5d45b3bf3bb4cfc85e2215)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:55.834348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba286541-8bc4-44e3-a243-d4556fd21025 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:55.899706 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca99457-673d-47d8-b857-f099aae52260 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:55.901148 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-dca99457-673d-47d8-b857-f099aae52260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:55.914148 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca99457-673d-47d8-b857-f099aae52260 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:55.918705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81a2d3-d43b-4cef-8adf-9806fbb6a8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:55.919565 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81a2d3-d43b-4cef-8adf-9806fbb6a8ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:55.919848 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81a2d3-d43b-4cef-8adf-9806fbb6a8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:55.920128 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81a2d3-d43b-4cef-8adf-9806fbb6a8ff admin admin] PATH_INFO: `/v3/users/65d22c693dd7409ba095afffc1494081` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:55.921318 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81a2d3-d43b-4cef-8adf-9806fbb6a8ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=65d22c693dd7409ba095afffc1494081)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:55.923343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81a2d3-d43b-4cef-8adf-9806fbb6a8ff admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:55.948011 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-ba286541-8bc4-44e3-a243-d4556fd21025 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d4ab182ab5d45b3bf3bb4cfc85e2215', 'request_id': 'req-ba286541-8bc4-44e3-a243-d4556fd21025'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:56.029643 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-ba286541-8bc4-44e3-a243-d4556fd21025 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d4ab182ab5d45b3bf3bb4cfc85e2215', 'request_id': 'req-ba286541-8bc4-44e3-a243-d4556fd21025'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:56.037563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-ba286541-8bc4-44e3-a243-d4556fd21025 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d4ab182ab5d45b3bf3bb4cfc85e2215', 'request_id': 'req-ba286541-8bc4-44e3-a243-d4556fd21025'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:56.049026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-ba286541-8bc4-44e3-a243-d4556fd21025 admin admin] The token cache is being invalidate because project 1d4ab182ab5d45b3bf3bb4cfc85e2215 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:45:56.051431 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-ba286541-8bc4-44e3-a243-d4556fd21025 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:56.051431 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-ba286541-8bc4-44e3-a243-d4556fd21025 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:56.051736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 965/1898] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:45:55 2022] DELETE /identity/v3/projects/1d4ab182ab5d45b3bf3bb4cfc85e2215 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:56.056865 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76d5f2-fda8-4abc-8842-59b90ba13f78 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:56.057154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76d5f2-fda8-4abc-8842-59b90ba13f78 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:56.057435 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76d5f2-fda8-4abc-8842-59b90ba13f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:56.076118 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-cf76d5f2-fda8-4abc-8842-59b90ba13f78 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:56.077334 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-cf76d5f2-fda8-4abc-8842-59b90ba13f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:56.105123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-4e81a2d3-d43b-4cef-8adf-9806fbb6a8ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65d22c693dd7409ba095afffc1494081'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:56.117818 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-4e81a2d3-d43b-4cef-8adf-9806fbb6a8ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65d22c693dd7409ba095afffc1494081'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:56.118793 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-4e81a2d3-d43b-4cef-8adf-9806fbb6a8ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65d22c693dd7409ba095afffc1494081'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:56.134621 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-4e81a2d3-d43b-4cef-8adf-9806fbb6a8ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65d22c693dd7409ba095afffc1494081'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:56.151519 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 966/1899] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:56.160227 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7652db-98f2-43cc-b939-8856e75f8162 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:56.160520 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7652db-98f2-43cc-b939-8856e75f8162 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:56.160840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7652db-98f2-43cc-b939-8856e75f8162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:56.180094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-cb7652db-98f2-43cc-b939-8856e75f8162 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:56.181322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-cb7652db-98f2-43cc-b939-8856e75f8162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:56.190189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 967/1900] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:45:56.193942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7652db-98f2-43cc-b939-8856e75f8162 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:56.201566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:56.205128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7652db-98f2-43cc-b939-8856e75f8162 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:56.208834 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-4e81a2d3-d43b-4cef-8adf-9806fbb6a8ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65d22c693dd7409ba095afffc1494081'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:56.211768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ac943-c882-45ec-b844-2a2a3660bb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:56.211996 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ac943-c882-45ec-b844-2a2a3660bb77 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:56.212334 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ac943-c882-45ec-b844-2a2a3660bb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:56.212685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ac943-c882-45ec-b844-2a2a3660bb77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:56.213493 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ac943-c882-45ec-b844-2a2a3660bb77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:45:56.213869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ac943-c882-45ec-b844-2a2a3660bb77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:56.215752 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ac943-c882-45ec-b844-2a2a3660bb77 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:56.217408 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 934/1901] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:45:55 2022] DELETE /identity/v3/users/65d22c693dd7409ba095afffc1494081 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:45:56.222195 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100ab07-28c1-467c-ad35-151a42a393f4 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:56.225948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100ab07-28c1-467c-ad35-151a42a393f4 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:56.225948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100ab07-28c1-467c-ad35-151a42a393f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:56.234634 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 968/1902] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:45:56.235730 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ac943-c882-45ec-b844-2a2a3660bb77 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:56.237470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-253ac943-c882-45ec-b844-2a2a3660bb77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:56.249740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-8100ab07-28c1-467c-ad35-151a42a393f4 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:56.251191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-8100ab07-28c1-467c-ad35-151a42a393f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:56.251480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ac943-c882-45ec-b844-2a2a3660bb77 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:56.256738 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:56.318341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149bc08-e284-42fa-a507-d858f691c5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:56.319261 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149bc08-e284-42fa-a507-d858f691c5fb admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:56.319653 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149bc08-e284-42fa-a507-d858f691c5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:56.320239 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149bc08-e284-42fa-a507-d858f691c5fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:56.321208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149bc08-e284-42fa-a507-d858f691c5fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:56.326022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149bc08-e284-42fa-a507-d858f691c5fb admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:56.327160 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 935/1903] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:45:56.333616 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100ab07-28c1-467c-ad35-151a42a393f4 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:56.335191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-8100ab07-28c1-467c-ad35-151a42a393f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:56.411304 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100ab07-28c1-467c-ad35-151a42a393f4 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:56.416616 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54857ab-0cc8-432c-a2b4-f631b4c835da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:56.417551 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54857ab-0cc8-432c-a2b4-f631b4c835da service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:56.417844 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54857ab-0cc8-432c-a2b4-f631b4c835da service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:56.418121 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54857ab-0cc8-432c-a2b4-f631b4c835da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:56.436210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 969/1904] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:56 2022] POST /identity/v3/projects => generated 387 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:56.437665 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54857ab-0cc8-432c-a2b4-f631b4c835da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:56.443835 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54857ab-0cc8-432c-a2b4-f631b4c835da service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:56.443975 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149bc08-e284-42fa-a507-d858f691c5fb admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:56.457508 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 936/1905] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:45:56.548689 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149bc08-e284-42fa-a507-d858f691c5fb admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:56.558177 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0c3e4-8375-435a-9f92-807be832c4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:56.559181 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0c3e4-8375-435a-9f92-807be832c4ef admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:56.559487 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0c3e4-8375-435a-9f92-807be832c4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:56.559760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0c3e4-8375-435a-9f92-807be832c4ef admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:56.560500 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe0c3e4-8375-435a-9f92-807be832c4ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:56.562513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe0c3e4-8375-435a-9f92-807be832c4ef admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:56.651836 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 970/1906] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:56 2022] POST /identity/v3/users => generated 294 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:56.655576 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54857ab-0cc8-432c-a2b4-f631b4c835da service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:56.667539 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54857ab-0cc8-432c-a2b4-f631b4c835da service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:56.673640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f394a79-6975-4ae6-9ed0-d601333a3d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:56.673640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f394a79-6975-4ae6-9ed0-d601333a3d47 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:56.674043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f394a79-6975-4ae6-9ed0-d601333a3d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:56.674333 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f394a79-6975-4ae6-9ed0-d601333a3d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:56.675099 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f394a79-6975-4ae6-9ed0-d601333a3d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:56.678617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f394a79-6975-4ae6-9ed0-d601333a3d47 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:56.690136 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 937/1907] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:56.693248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0c3e4-8375-435a-9f92-807be832c4ef admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:56.714523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0c3e4-8375-435a-9f92-807be832c4ef admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:56.720087 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973b9c4-5672-44ab-be62-836b8e84a206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:56.720768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973b9c4-5672-44ab-be62-836b8e84a206 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:56.721059 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973b9c4-5672-44ab-be62-836b8e84a206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:56.721399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973b9c4-5672-44ab-be62-836b8e84a206 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:56.722249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d973b9c4-5672-44ab-be62-836b8e84a206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:56.724624 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d973b9c4-5672-44ab-be62-836b8e84a206 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:56.742728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 971/1908] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:56.743900 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f394a79-6975-4ae6-9ed0-d601333a3d47 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:56.761489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f394a79-6975-4ae6-9ed0-d601333a3d47 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:56.772794 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dde8a5-331b-4b20-adb8-6e41fc9a6bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:56.773695 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dde8a5-331b-4b20-adb8-6e41fc9a6bf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:56.773981 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dde8a5-331b-4b20-adb8-6e41fc9a6bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:56.774329 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dde8a5-331b-4b20-adb8-6e41fc9a6bf4 admin admin] PATH_INFO: `/v3/projects/aaea08d59f49469f9d6925a253f5ca64/users/7998cea465244a6b946c8a50e1b9652e/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:56.798771 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dde8a5-331b-4b20-adb8-6e41fc9a6bf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaea08d59f49469f9d6925a253f5ca64, user_id=7998cea465244a6b946c8a50e1b9652e, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:56.802008 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dde8a5-331b-4b20-adb8-6e41fc9a6bf4 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:56.869809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 938/1909] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:56 2022] PUT /identity/v3/projects/aaea08d59f49469f9d6925a253f5ca64/users/7998cea465244a6b946c8a50e1b9652e/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:56.882278 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8178be-ab03-4589-b8a7-b851cdd12a1c None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:56.882909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8178be-ab03-4589-b8a7-b851cdd12a1c None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:56.883520 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8178be-ab03-4589-b8a7-b851cdd12a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:56.914537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-ce8178be-ab03-4589-b8a7-b851cdd12a1c None None] MFA Rules not processed for user `7998cea465244a6b946c8a50e1b9652e`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:56.915765 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ce8178be-ab03-4589-b8a7-b851cdd12a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:56.968854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:57.005226 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 972/1910] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:45:57.011598 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dde8a5-331b-4b20-adb8-6e41fc9a6bf4 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:57.096625 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dde8a5-331b-4b20-adb8-6e41fc9a6bf4 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:57.102812 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14282511-0bdb-433d-ae4f-368d30759ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:57.103474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14282511-0bdb-433d-ae4f-368d30759ce9 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:57.103757 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14282511-0bdb-433d-ae4f-368d30759ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:57.104036 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14282511-0bdb-433d-ae4f-368d30759ce9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:57.104726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14282511-0bdb-433d-ae4f-368d30759ce9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:57.106656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14282511-0bdb-433d-ae4f-368d30759ce9 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:57.169662 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8178be-ab03-4589-b8a7-b851cdd12a1c None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:57.171073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ce8178be-ab03-4589-b8a7-b851cdd12a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:57.185730 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8178be-ab03-4589-b8a7-b851cdd12a1c None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:57.191884 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18de003-ac01-44fe-8bd3-537cdefd306f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:57.192904 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18de003-ac01-44fe-8bd3-537cdefd306f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:57.193188 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18de003-ac01-44fe-8bd3-537cdefd306f admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:57.193498 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18de003-ac01-44fe-8bd3-537cdefd306f admin admin] PATH_INFO: `/v3/projects/bec2f9b1e8d048de96626c8ba3426eec` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:57.196472 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18de003-ac01-44fe-8bd3-537cdefd306f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bec2f9b1e8d048de96626c8ba3426eec)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:57.199308 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18de003-ac01-44fe-8bd3-537cdefd306f admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:57.201010 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 939/1911] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:57 2022] POST /identity/v3/projects => generated 387 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:57.205798 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14282511-0bdb-433d-ae4f-368d30759ce9 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:57.215033 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:57.280622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14282511-0bdb-433d-ae4f-368d30759ce9 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:57.285238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9a5d6-1835-4cb6-87b6-bbb2964c1ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:57.285901 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9a5d6-1835-4cb6-87b6-bbb2964c1ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:57.286183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9a5d6-1835-4cb6-87b6-bbb2964c1ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:57.286485 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9a5d6-1835-4cb6-87b6-bbb2964c1ff2 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:57.287168 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda9a5d6-1835-4cb6-87b6-bbb2964c1ff2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:57.289142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda9a5d6-1835-4cb6-87b6-bbb2964c1ff2 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:57.345256 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-e18de003-ac01-44fe-8bd3-537cdefd306f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bec2f9b1e8d048de96626c8ba3426eec', 'request_id': 'req-e18de003-ac01-44fe-8bd3-537cdefd306f'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:57.424038 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 940/1912] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:57 2022] POST /identity/v3/users => generated 300 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:45:57.427772 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9a5d6-1835-4cb6-87b6-bbb2964c1ff2 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:57.439957 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9a5d6-1835-4cb6-87b6-bbb2964c1ff2 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:57.463724 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-e18de003-ac01-44fe-8bd3-537cdefd306f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bec2f9b1e8d048de96626c8ba3426eec', 'request_id': 'req-e18de003-ac01-44fe-8bd3-537cdefd306f'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:57.470576 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-e18de003-ac01-44fe-8bd3-537cdefd306f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bec2f9b1e8d048de96626c8ba3426eec', 'request_id': 'req-e18de003-ac01-44fe-8bd3-537cdefd306f'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:57.481255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-e18de003-ac01-44fe-8bd3-537cdefd306f admin admin] The token cache is being invalidate because project bec2f9b1e8d048de96626c8ba3426eec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:45:57.481598 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-e18de003-ac01-44fe-8bd3-537cdefd306f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:57.482546 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-e18de003-ac01-44fe-8bd3-537cdefd306f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:57.484317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 973/1913] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:45:57 2022] DELETE /identity/v3/projects/bec2f9b1e8d048de96626c8ba3426eec => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:45:57.488019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18de003-ac01-44fe-8bd3-537cdefd306f admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:57.489320 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-e18de003-ac01-44fe-8bd3-537cdefd306f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:57.500130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6015cf-8a2a-404c-b9db-51975b430f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:57.500796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6015cf-8a2a-404c-b9db-51975b430f81 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:57.501082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6015cf-8a2a-404c-b9db-51975b430f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:57.501360 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6015cf-8a2a-404c-b9db-51975b430f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:57.502110 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6015cf-8a2a-404c-b9db-51975b430f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:57.503771 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6015cf-8a2a-404c-b9db-51975b430f81 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:57.517482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 941/1914] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:45:57.518513 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6015cf-8a2a-404c-b9db-51975b430f81 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:57.520243 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-1c6015cf-8a2a-404c-b9db-51975b430f81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:57.535380 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6015cf-8a2a-404c-b9db-51975b430f81 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:57.538502 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dd0e7d-6a95-4acb-8558-1d427037e9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:57.539528 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd0e7d-6a95-4acb-8558-1d427037e9f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:57.539811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd0e7d-6a95-4acb-8558-1d427037e9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:57.540090 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd0e7d-6a95-4acb-8558-1d427037e9f7 admin admin] PATH_INFO: `/v3/projects/7e4ccfa1395348d0bd278abb003e2437/users/e2e3ef512bc8476fb9ad20639ade3696/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:57.556540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd0e7d-6a95-4acb-8558-1d427037e9f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e4ccfa1395348d0bd278abb003e2437, user_id=e2e3ef512bc8476fb9ad20639ade3696, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:57.559396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18de003-ac01-44fe-8bd3-537cdefd306f admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:57.559589 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd0e7d-6a95-4acb-8558-1d427037e9f7 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:57.564896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a365f7-b144-4d96-aa27-eae5de9c23fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:57.565822 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a365f7-b144-4d96-aa27-eae5de9c23fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:57.566169 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a365f7-b144-4d96-aa27-eae5de9c23fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:57.566514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a365f7-b144-4d96-aa27-eae5de9c23fd admin admin] PATH_INFO: `/v3/users/97b06217e56142ceaf04da1c772f33a4` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:57.567754 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a365f7-b144-4d96-aa27-eae5de9c23fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=97b06217e56142ceaf04da1c772f33a4)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:57.570365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a365f7-b144-4d96-aa27-eae5de9c23fd admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:57.624226 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 942/1915] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:57 2022] PUT /identity/v3/projects/7e4ccfa1395348d0bd278abb003e2437/users/e2e3ef512bc8476fb9ad20639ade3696/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:57.628124 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dd0e7d-6a95-4acb-8558-1d427037e9f7 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:57.689621 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dd0e7d-6a95-4acb-8558-1d427037e9f7 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:57.695300 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71950e2e-875a-43ea-aec6-5f52d00dc129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:57.695978 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71950e2e-875a-43ea-aec6-5f52d00dc129 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:57.696269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71950e2e-875a-43ea-aec6-5f52d00dc129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:57.696641 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71950e2e-875a-43ea-aec6-5f52d00dc129 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:57.697383 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71950e2e-875a-43ea-aec6-5f52d00dc129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:57.699069 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71950e2e-875a-43ea-aec6-5f52d00dc129 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:57.709899 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 943/1916] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:45:57.714100 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71950e2e-875a-43ea-aec6-5f52d00dc129 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:57.723616 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71950e2e-875a-43ea-aec6-5f52d00dc129 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:57.728962 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446b659-0abe-478e-af0e-26f982895bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:57.729806 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446b659-0abe-478e-af0e-26f982895bd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:57.730083 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446b659-0abe-478e-af0e-26f982895bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:57.730356 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446b659-0abe-478e-af0e-26f982895bd3 admin admin] PATH_INFO: `/v3/projects/7e4ccfa1395348d0bd278abb003e2437/users/e2e3ef512bc8476fb9ad20639ade3696/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:57.732562 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5446b659-0abe-478e-af0e-26f982895bd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e4ccfa1395348d0bd278abb003e2437, user_id=e2e3ef512bc8476fb9ad20639ade3696, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:57.735219 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5446b659-0abe-478e-af0e-26f982895bd3 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:57.746894 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 944/1917] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:57 2022] PUT /identity/v3/projects/7e4ccfa1395348d0bd278abb003e2437/users/e2e3ef512bc8476fb9ad20639ade3696/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:45:57.757102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db007270-9a65-4d58-91f1-e57d1a182730 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:57.757386 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db007270-9a65-4d58-91f1-e57d1a182730 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:57.757664 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db007270-9a65-4d58-91f1-e57d1a182730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:57.777614 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f4a365f7-b144-4d96-aa27-eae5de9c23fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97b06217e56142ceaf04da1c772f33a4'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:57.784736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f4a365f7-b144-4d96-aa27-eae5de9c23fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97b06217e56142ceaf04da1c772f33a4'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:57.784996 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-db007270-9a65-4d58-91f1-e57d1a182730 None None] MFA Rules not processed for user `e2e3ef512bc8476fb9ad20639ade3696`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:57.785426 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f4a365f7-b144-4d96-aa27-eae5de9c23fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97b06217e56142ceaf04da1c772f33a4'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:57.786052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f4a365f7-b144-4d96-aa27-eae5de9c23fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97b06217e56142ceaf04da1c772f33a4'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:57.788095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:57.791928 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-db007270-9a65-4d58-91f1-e57d1a182730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:57.797349 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:57.799643 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f4a365f7-b144-4d96-aa27-eae5de9c23fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97b06217e56142ceaf04da1c772f33a4'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:57.877338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 945/1918] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:45:57.887520 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 974/1919] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:45:57 2022] DELETE /identity/v3/users/97b06217e56142ceaf04da1c772f33a4 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:45:57.888875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db007270-9a65-4d58-91f1-e57d1a182730 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:57.891431 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-db007270-9a65-4d58-91f1-e57d1a182730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:57.954910 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db007270-9a65-4d58-91f1-e57d1a182730 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:58.004681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317492d-1271-4259-b2ba-da38071f0dc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:58.005366 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317492d-1271-4259-b2ba-da38071f0dc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:58.005653 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317492d-1271-4259-b2ba-da38071f0dc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:58.005930 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317492d-1271-4259-b2ba-da38071f0dc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:58.054319 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:58.073138 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317492d-1271-4259-b2ba-da38071f0dc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:58.074603 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317492d-1271-4259-b2ba-da38071f0dc6 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:58.086325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 946/1920] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:45:58.190758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a365f7-b144-4d96-aa27-eae5de9c23fd admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:58.195602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-f4a365f7-b144-4d96-aa27-eae5de9c23fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:58.277745 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a365f7-b144-4d96-aa27-eae5de9c23fd admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:58.284499 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a097fd-da60-4b74-aee1-ff1a9cf95ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:58.285382 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a097fd-da60-4b74-aee1-ff1a9cf95ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:58.285661 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a097fd-da60-4b74-aee1-ff1a9cf95ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:58.285939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a097fd-da60-4b74-aee1-ff1a9cf95ed8 admin admin] PATH_INFO: `/v3/users/7998cea465244a6b946c8a50e1b9652e` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:58.287443 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a097fd-da60-4b74-aee1-ff1a9cf95ed8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7998cea465244a6b946c8a50e1b9652e)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:58.289548 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a097fd-da60-4b74-aee1-ff1a9cf95ed8 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:58.405395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317492d-1271-4259-b2ba-da38071f0dc6 service cinder] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:58.407102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-2317492d-1271-4259-b2ba-da38071f0dc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:58.422937 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317492d-1271-4259-b2ba-da38071f0dc6 service cinder] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:58.427760 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841021b5-3219-45d8-81b5-71020a1000f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:58.429250 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841021b5-3219-45d8-81b5-71020a1000f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:58.429531 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841021b5-3219-45d8-81b5-71020a1000f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:58.430231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841021b5-3219-45d8-81b5-71020a1000f6 admin admin] PATH_INFO: `/v3/projects/31bc81f62bd343bbb24ded4412e8206e` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:58.431203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841021b5-3219-45d8-81b5-71020a1000f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31bc81f62bd343bbb24ded4412e8206e)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:58.433299 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841021b5-3219-45d8-81b5-71020a1000f6 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:58.508776 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-48a097fd-da60-4b74-aee1-ff1a9cf95ed8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7998cea465244a6b946c8a50e1b9652e'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:58.520477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-48a097fd-da60-4b74-aee1-ff1a9cf95ed8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7998cea465244a6b946c8a50e1b9652e'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:58.521161 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-48a097fd-da60-4b74-aee1-ff1a9cf95ed8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7998cea465244a6b946c8a50e1b9652e'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:58.524941 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-48a097fd-da60-4b74-aee1-ff1a9cf95ed8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7998cea465244a6b946c8a50e1b9652e'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:58.534808 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-48a097fd-da60-4b74-aee1-ff1a9cf95ed8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7998cea465244a6b946c8a50e1b9652e'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:58.539821 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-841021b5-3219-45d8-81b5-71020a1000f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31bc81f62bd343bbb24ded4412e8206e', 'request_id': 'req-841021b5-3219-45d8-81b5-71020a1000f6'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:58.600807 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 975/1921] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:45:58 2022] DELETE /identity/v3/users/7998cea465244a6b946c8a50e1b9652e => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:58.605091 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0c22c-13f5-442f-a71b-76d42f68971c None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:58.605465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0c22c-13f5-442f-a71b-76d42f68971c None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:58.605770 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0c22c-13f5-442f-a71b-76d42f68971c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:58.622440 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-d5c0c22c-13f5-442f-a71b-76d42f68971c None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:58.623638 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-d5c0c22c-13f5-442f-a71b-76d42f68971c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:58.640827 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-841021b5-3219-45d8-81b5-71020a1000f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31bc81f62bd343bbb24ded4412e8206e', 'request_id': 'req-841021b5-3219-45d8-81b5-71020a1000f6'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:58.648696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-841021b5-3219-45d8-81b5-71020a1000f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31bc81f62bd343bbb24ded4412e8206e', 'request_id': 'req-841021b5-3219-45d8-81b5-71020a1000f6'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:58.660621 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-841021b5-3219-45d8-81b5-71020a1000f6 admin admin] The token cache is being invalidate because project 31bc81f62bd343bbb24ded4412e8206e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:45:58.660621 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-841021b5-3219-45d8-81b5-71020a1000f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:58.660928 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-841021b5-3219-45d8-81b5-71020a1000f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:58.662566 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 947/1922] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:45:58 2022] DELETE /identity/v3/projects/31bc81f62bd343bbb24ded4412e8206e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:45:58.667519 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44744d52-f03b-409c-9d2b-7ee578dddc5c None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:58.667832 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44744d52-f03b-409c-9d2b-7ee578dddc5c None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:58.668114 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44744d52-f03b-409c-9d2b-7ee578dddc5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:58.685078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 976/1923] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:45:58.688315 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-44744d52-f03b-409c-9d2b-7ee578dddc5c None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:58.689391 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-44744d52-f03b-409c-9d2b-7ee578dddc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:58.695259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0c22c-13f5-442f-a71b-76d42f68971c None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:58.696565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-d5c0c22c-13f5-442f-a71b-76d42f68971c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:58.698138 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 948/1924] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:45:58.705652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91354a8-a665-4be6-9867-d357c0397e75 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:58.706033 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91354a8-a665-4be6-9867-d357c0397e75 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:58.706321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91354a8-a665-4be6-9867-d357c0397e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:58.706806 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:58.724767 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-d91354a8-a665-4be6-9867-d357c0397e75 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:58.726091 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-d91354a8-a665-4be6-9867-d357c0397e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:58.734543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 949/1925] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:45:58.739036 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91354a8-a665-4be6-9867-d357c0397e75 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:58.749118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91354a8-a665-4be6-9867-d357c0397e75 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:58.754163 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcbcb5b-508c-44c1-aff1-9b243974a7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:58.754864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbcb5b-508c-44c1-aff1-9b243974a7f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:58.755144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbcb5b-508c-44c1-aff1-9b243974a7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:58.755417 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbcb5b-508c-44c1-aff1-9b243974a7f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:58.756654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcbcb5b-508c-44c1-aff1-9b243974a7f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:45:58.756972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcbcb5b-508c-44c1-aff1-9b243974a7f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:58.758835 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcbcb5b-508c-44c1-aff1-9b243974a7f0 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:58.759683 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0c22c-13f5-442f-a71b-76d42f68971c None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:58.764090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d450197-e861-440d-85cd-70cb4376631f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:58.764736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d450197-e861-440d-85cd-70cb4376631f service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:58.765012 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d450197-e861-440d-85cd-70cb4376631f service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:58.765289 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d450197-e861-440d-85cd-70cb4376631f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:58.771101 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 950/1926] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:45:58.776114 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcbcb5b-508c-44c1-aff1-9b243974a7f0 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:58.778670 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d450197-e861-440d-85cd-70cb4376631f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:58.780356 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d450197-e861-440d-85cd-70cb4376631f service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:58.785597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcbcb5b-508c-44c1-aff1-9b243974a7f0 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:58.790582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 977/1927] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:45:58.791963 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec8246-faf1-4b11-a866-917e9d68a21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:58.791963 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec8246-faf1-4b11-a866-917e9d68a21b admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:58.791963 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec8246-faf1-4b11-a866-917e9d68a21b admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:58.792206 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec8246-faf1-4b11-a866-917e9d68a21b admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:58.792757 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec8246-faf1-4b11-a866-917e9d68a21b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:58.794756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec8246-faf1-4b11-a866-917e9d68a21b admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:58.887682 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 951/1928] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:45:58 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:45:58.891358 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d450197-e861-440d-85cd-70cb4376631f service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:58.902502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d450197-e861-440d-85cd-70cb4376631f service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:58.960506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f11483-ec54-45c5-91d6-dd1ed118501d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:58.961186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f11483-ec54-45c5-91d6-dd1ed118501d admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:58.961473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f11483-ec54-45c5-91d6-dd1ed118501d admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:58.961753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f11483-ec54-45c5-91d6-dd1ed118501d admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:58.962494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f11483-ec54-45c5-91d6-dd1ed118501d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:58.964494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f11483-ec54-45c5-91d6-dd1ed118501d admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:59.059522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 978/1929] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:45:58 2022] POST /identity/v3/users => generated 292 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:45:59.063533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec8246-faf1-4b11-a866-917e9d68a21b admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:59.074637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec8246-faf1-4b11-a866-917e9d68a21b admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:59.079452 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763aa61c-6325-4fe1-af93-25ff90715fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:59.080193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aa61c-6325-4fe1-af93-25ff90715fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:59.080608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aa61c-6325-4fe1-af93-25ff90715fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:59.080936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aa61c-6325-4fe1-af93-25ff90715fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:59.081666 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763aa61c-6325-4fe1-af93-25ff90715fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:59.083317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763aa61c-6325-4fe1-af93-25ff90715fc3 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:59.094455 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 952/1930] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:45:59.098411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f11483-ec54-45c5-91d6-dd1ed118501d admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:59.110059 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f11483-ec54-45c5-91d6-dd1ed118501d admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:59.114859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1783c-378e-4a04-99d3-d71372956477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:59.115579 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1783c-378e-4a04-99d3-d71372956477 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:59.116032 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1783c-378e-4a04-99d3-d71372956477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:59.116704 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1783c-378e-4a04-99d3-d71372956477 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:59.117047 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1783c-378e-4a04-99d3-d71372956477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:59.118711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1783c-378e-4a04-99d3-d71372956477 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:59.129135 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 979/1931] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:45:59.133777 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763aa61c-6325-4fe1-af93-25ff90715fc3 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:59.143651 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763aa61c-6325-4fe1-af93-25ff90715fc3 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:59.148903 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ac74c-1adc-4f8c-bf1b-c72fb769afa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:59.149776 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ac74c-1adc-4f8c-bf1b-c72fb769afa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:59.150041 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ac74c-1adc-4f8c-bf1b-c72fb769afa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:59.150315 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ac74c-1adc-4f8c-bf1b-c72fb769afa0 admin admin] PATH_INFO: `/v3/projects/f7ebbaa0a00b41bd8af1a4401c392193/users/001c6fdcc55c421e82a6e3f5c1a36fdf/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:59.164753 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ac74c-1adc-4f8c-bf1b-c72fb769afa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7ebbaa0a00b41bd8af1a4401c392193, user_id=001c6fdcc55c421e82a6e3f5c1a36fdf, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:59.167471 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ac74c-1adc-4f8c-bf1b-c72fb769afa0 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:59.202806 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1783c-378e-4a04-99d3-d71372956477 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:59.204601 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-58e1783c-378e-4a04-99d3-d71372956477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:59.224327 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1783c-378e-4a04-99d3-d71372956477 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:59.229060 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 953/1932] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:45:59 2022] PUT /identity/v3/projects/f7ebbaa0a00b41bd8af1a4401c392193/users/001c6fdcc55c421e82a6e3f5c1a36fdf/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:45:59.241252 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a105c6-cea7-460e-b7c8-506b8784df91 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:59.241678 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a105c6-cea7-460e-b7c8-506b8784df91 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:59.242037 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a105c6-cea7-460e-b7c8-506b8784df91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:59.275801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-16a105c6-cea7-460e-b7c8-506b8784df91 None None] MFA Rules not processed for user `001c6fdcc55c421e82a6e3f5c1a36fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:45:59.277021 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-16a105c6-cea7-460e-b7c8-506b8784df91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:59.306979 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:59.309244 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6deeb-7e38-41f4-a713-ce1b11a83e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:59.310260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6deeb-7e38-41f4-a713-ce1b11a83e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:59.310575 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6deeb-7e38-41f4-a713-ce1b11a83e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:59.310856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6deeb-7e38-41f4-a713-ce1b11a83e27 admin admin] PATH_INFO: `/v3/projects/aaea08d59f49469f9d6925a253f5ca64` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:59.312048 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6deeb-7e38-41f4-a713-ce1b11a83e27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaea08d59f49469f9d6925a253f5ca64)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:59.314640 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6deeb-7e38-41f4-a713-ce1b11a83e27 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:59.352638 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 954/1933] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:45:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:45:59.363766 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a105c6-cea7-460e-b7c8-506b8784df91 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:59.365278 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-16a105c6-cea7-460e-b7c8-506b8784df91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:45:59.426241 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a105c6-cea7-460e-b7c8-506b8784df91 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:59.430303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9f21e-8756-4f5c-bf13-570cea316fa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:59.430983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9f21e-8756-4f5c-bf13-570cea316fa3 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:59.431647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9f21e-8756-4f5c-bf13-570cea316fa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:59.431647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9f21e-8756-4f5c-bf13-570cea316fa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:59.440903 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9f21e-8756-4f5c-bf13-570cea316fa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:59.442663 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9f21e-8756-4f5c-bf13-570cea316fa3 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:59.457268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 955/1934] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:45:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:45:59.468821 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:45:59.487500 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-13e6deeb-7e38-41f4-a713-ce1b11a83e27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaea08d59f49469f9d6925a253f5ca64', 'request_id': 'req-13e6deeb-7e38-41f4-a713-ce1b11a83e27'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:59.574568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-13e6deeb-7e38-41f4-a713-ce1b11a83e27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaea08d59f49469f9d6925a253f5ca64', 'request_id': 'req-13e6deeb-7e38-41f4-a713-ce1b11a83e27'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:59.583588 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-13e6deeb-7e38-41f4-a713-ce1b11a83e27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaea08d59f49469f9d6925a253f5ca64', 'request_id': 'req-13e6deeb-7e38-41f4-a713-ce1b11a83e27'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:59.595967 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-13e6deeb-7e38-41f4-a713-ce1b11a83e27 admin admin] The token cache is being invalidate because project aaea08d59f49469f9d6925a253f5ca64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:45:59.596262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-13e6deeb-7e38-41f4-a713-ce1b11a83e27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:59.596967 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-13e6deeb-7e38-41f4-a713-ce1b11a83e27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:59.598609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 980/1935] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:45:59 2022] DELETE /identity/v3/projects/aaea08d59f49469f9d6925a253f5ca64 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:45:59.602886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9f21e-8756-4f5c-bf13-570cea316fa3 service cinder] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:45:59.618104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9f21e-8756-4f5c-bf13-570cea316fa3 service cinder] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:45:59.690142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c0e03-905a-453d-a124-560848b670e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:45:59.691058 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c0e03-905a-453d-a124-560848b670e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:45:59.691341 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c0e03-905a-453d-a124-560848b670e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:45:59.691614 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c0e03-905a-453d-a124-560848b670e6 admin admin] PATH_INFO: `/v3/users/e2e3ef512bc8476fb9ad20639ade3696` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:45:59.692830 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c0e03-905a-453d-a124-560848b670e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2e3ef512bc8476fb9ad20639ade3696)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:45:59.695674 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c0e03-905a-453d-a124-560848b670e6 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:45:59.918335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-589c0e03-905a-453d-a124-560848b670e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2e3ef512bc8476fb9ad20639ade3696'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:59.927671 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-589c0e03-905a-453d-a124-560848b670e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2e3ef512bc8476fb9ad20639ade3696'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:59.928953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-589c0e03-905a-453d-a124-560848b670e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2e3ef512bc8476fb9ad20639ade3696'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:59.943800 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-589c0e03-905a-453d-a124-560848b670e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2e3ef512bc8476fb9ad20639ade3696'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:45:59.972614 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:00.019766 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-589c0e03-905a-453d-a124-560848b670e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2e3ef512bc8476fb9ad20639ade3696'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:00.026714 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 956/1936] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:45:59 2022] DELETE /identity/v3/users/e2e3ef512bc8476fb9ad20639ade3696 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:46:00.477262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6deeb-7e38-41f4-a713-ce1b11a83e27 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:00.479400 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-13e6deeb-7e38-41f4-a713-ce1b11a83e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:00.508429 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6deeb-7e38-41f4-a713-ce1b11a83e27 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:00.580670 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896bc39-b6fa-423b-917b-c423d4f267ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:00.581706 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896bc39-b6fa-423b-917b-c423d4f267ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:00.582049 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896bc39-b6fa-423b-917b-c423d4f267ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:00.582487 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896bc39-b6fa-423b-917b-c423d4f267ff admin admin] PATH_INFO: `/v3/projects/7e4ccfa1395348d0bd278abb003e2437` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:00.583976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7896bc39-b6fa-423b-917b-c423d4f267ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e4ccfa1395348d0bd278abb003e2437)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:00.586527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7896bc39-b6fa-423b-917b-c423d4f267ff admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:00.626806 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac510fa4-d1d1-4a26-b5cf-fe1f3a31906b None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:00.627638 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac510fa4-d1d1-4a26-b5cf-fe1f3a31906b None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:00.628087 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac510fa4-d1d1-4a26-b5cf-fe1f3a31906b None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:00.630110 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 957/1937] 10.222.0.15 () {54 vars in 921 bytes} [Mon Jun 27 18:46:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 18:46:00.645720 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14866d3-c168-4880-8713-3a1c2b6a65cc None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:00.648084 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14866d3-c168-4880-8713-3a1c2b6a65cc None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:00.648434 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14866d3-c168-4880-8713-3a1c2b6a65cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:00.677825 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-f14866d3-c168-4880-8713-3a1c2b6a65cc None None] MFA Rules not processed for user `1dddd4efb44c4152bb4151f7715f2223`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:00.680307 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f14866d3-c168-4880-8713-3a1c2b6a65cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:00.786335 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7896bc39-b6fa-423b-917b-c423d4f267ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e4ccfa1395348d0bd278abb003e2437', 'request_id': 'req-7896bc39-b6fa-423b-917b-c423d4f267ff'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:00.807961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 958/1938] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Jun 27 18:46:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:00.886874 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7896bc39-b6fa-423b-917b-c423d4f267ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e4ccfa1395348d0bd278abb003e2437', 'request_id': 'req-7896bc39-b6fa-423b-917b-c423d4f267ff'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:00.901758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7896bc39-b6fa-423b-917b-c423d4f267ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e4ccfa1395348d0bd278abb003e2437', 'request_id': 'req-7896bc39-b6fa-423b-917b-c423d4f267ff'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:00.917930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7896bc39-b6fa-423b-917b-c423d4f267ff admin admin] The token cache is being invalidate because project 7e4ccfa1395348d0bd278abb003e2437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:46:00.918234 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7896bc39-b6fa-423b-917b-c423d4f267ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:00.919001 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7896bc39-b6fa-423b-917b-c423d4f267ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:00.920911 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 981/1939] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:46:00 2022] DELETE /identity/v3/projects/7e4ccfa1395348d0bd278abb003e2437 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:00.925671 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b75436-436c-43e1-8f6c-5c1c1d4fd0f8 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:00.925965 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b75436-436c-43e1-8f6c-5c1c1d4fd0f8 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:00.926245 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b75436-436c-43e1-8f6c-5c1c1d4fd0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:00.958333 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-79b75436-436c-43e1-8f6c-5c1c1d4fd0f8 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:00.965273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-79b75436-436c-43e1-8f6c-5c1c1d4fd0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:01.006229 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896bc39-b6fa-423b-917b-c423d4f267ff admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:01.008987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-7896bc39-b6fa-423b-917b-c423d4f267ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:01.103533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 959/1940] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:01.113748 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0fb3e-86d9-411d-a4bd-f47193024479 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:01.114370 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0fb3e-86d9-411d-a4bd-f47193024479 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:01.114837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0fb3e-86d9-411d-a4bd-f47193024479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:01.122919 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:01.136693 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-d9a0fb3e-86d9-411d-a4bd-f47193024479 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:01.138145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-d9a0fb3e-86d9-411d-a4bd-f47193024479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:01.148401 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 960/1941] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:01.152494 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0fb3e-86d9-411d-a4bd-f47193024479 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:01.160482 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896bc39-b6fa-423b-917b-c423d4f267ff admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:01.164185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0fb3e-86d9-411d-a4bd-f47193024479 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:01.169616 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136aa888-4eb9-46e8-bae3-676ac0d9cc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:01.169776 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46851a7f-4de9-453a-a62c-c28f1d4965cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:01.170321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136aa888-4eb9-46e8-bae3-676ac0d9cc3c admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:01.170580 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46851a7f-4de9-453a-a62c-c28f1d4965cf service nova] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:01.170706 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136aa888-4eb9-46e8-bae3-676ac0d9cc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:01.170870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46851a7f-4de9-453a-a62c-c28f1d4965cf service nova] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:01.170973 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136aa888-4eb9-46e8-bae3-676ac0d9cc3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:01.171149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46851a7f-4de9-453a-a62c-c28f1d4965cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:01.171914 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136aa888-4eb9-46e8-bae3-676ac0d9cc3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:46:01.172225 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136aa888-4eb9-46e8-bae3-676ac0d9cc3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:01.173090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-46851a7f-4de9-453a-a62c-c28f1d4965cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:01.173886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136aa888-4eb9-46e8-bae3-676ac0d9cc3c admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:01.190145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 961/1942] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:46:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:46:01.192674 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46851a7f-4de9-453a-a62c-c28f1d4965cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:01.196352 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136aa888-4eb9-46e8-bae3-676ac0d9cc3c admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:01.199045 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46851a7f-4de9-453a-a62c-c28f1d4965cf service nova] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:01.212556 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136aa888-4eb9-46e8-bae3-676ac0d9cc3c admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:01.230175 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 982/1943] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:01.249236 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1924bb7-3f8f-4370-b39e-cdbd7f8c26f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:01.250072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1924bb7-3f8f-4370-b39e-cdbd7f8c26f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:01.250072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1924bb7-3f8f-4370-b39e-cdbd7f8c26f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:01.250601 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1924bb7-3f8f-4370-b39e-cdbd7f8c26f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:01.254141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1924bb7-3f8f-4370-b39e-cdbd7f8c26f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:01.256560 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1924bb7-3f8f-4370-b39e-cdbd7f8c26f7 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:01.394160 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 962/1944] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:46:01 2022] POST /identity/v3/projects => generated 383 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:01.395317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46851a7f-4de9-453a-a62c-c28f1d4965cf service nova] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:01.407797 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46851a7f-4de9-453a-a62c-c28f1d4965cf service nova] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:01.499565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c7ef4-211d-4cad-a8c4-0d1274e86b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:01.501713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c7ef4-211d-4cad-a8c4-0d1274e86b8c admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:01.501713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c7ef4-211d-4cad-a8c4-0d1274e86b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:01.501713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c7ef4-211d-4cad-a8c4-0d1274e86b8c admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:01.501713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c7ef4-211d-4cad-a8c4-0d1274e86b8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:01.503761 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c7ef4-211d-4cad-a8c4-0d1274e86b8c admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:01.620844 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 983/1945] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:46:01 2022] POST /identity/v3/users => generated 292 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:46:01.625469 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1924bb7-3f8f-4370-b39e-cdbd7f8c26f7 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:01.650040 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1924bb7-3f8f-4370-b39e-cdbd7f8c26f7 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:01.656492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8dfcdb-89e4-42c8-8948-cbbce217fa03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:01.657033 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dfcdb-89e4-42c8-8948-cbbce217fa03 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:01.657319 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dfcdb-89e4-42c8-8948-cbbce217fa03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:01.658165 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dfcdb-89e4-42c8-8948-cbbce217fa03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:01.658572 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8dfcdb-89e4-42c8-8948-cbbce217fa03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:01.660456 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8dfcdb-89e4-42c8-8948-cbbce217fa03 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:01.673887 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 963/1946] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:01.675688 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c7ef4-211d-4cad-a8c4-0d1274e86b8c admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:01.687249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c7ef4-211d-4cad-a8c4-0d1274e86b8c admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:01.692153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3a8ef-ce04-46ae-9f63-d9e20f40857d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:01.692845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3a8ef-ce04-46ae-9f63-d9e20f40857d admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:01.693124 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3a8ef-ce04-46ae-9f63-d9e20f40857d admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:01.693400 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3a8ef-ce04-46ae-9f63-d9e20f40857d admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:01.694792 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a3a8ef-ce04-46ae-9f63-d9e20f40857d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:01.696535 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a3a8ef-ce04-46ae-9f63-d9e20f40857d admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:01.708117 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 984/1947] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:46:01.714203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8dfcdb-89e4-42c8-8948-cbbce217fa03 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:01.724416 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8dfcdb-89e4-42c8-8948-cbbce217fa03 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:01.747055 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86ca1a-a8a6-4fc6-ba7e-8bae23f82b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:01.748274 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86ca1a-a8a6-4fc6-ba7e-8bae23f82b03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:01.748274 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86ca1a-a8a6-4fc6-ba7e-8bae23f82b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:01.748476 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86ca1a-a8a6-4fc6-ba7e-8bae23f82b03 admin admin] PATH_INFO: `/v3/projects/031c5509bf994c0b889910d29051ebec/users/e075cd751e554b64baae5cf69eb1e344/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:01.765161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86ca1a-a8a6-4fc6-ba7e-8bae23f82b03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=031c5509bf994c0b889910d29051ebec, user_id=e075cd751e554b64baae5cf69eb1e344, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:01.767894 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86ca1a-a8a6-4fc6-ba7e-8bae23f82b03 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:01.821498 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 964/1948] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:01 2022] PUT /identity/v3/projects/031c5509bf994c0b889910d29051ebec/users/e075cd751e554b64baae5cf69eb1e344/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:01.833634 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b73d30-8516-4311-940e-128910781c99 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:01.834058 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b73d30-8516-4311-940e-128910781c99 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:01.834483 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b73d30-8516-4311-940e-128910781c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:01.861395 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-c4b73d30-8516-4311-940e-128910781c99 None None] MFA Rules not processed for user `e075cd751e554b64baae5cf69eb1e344`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:01.862710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c4b73d30-8516-4311-940e-128910781c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:01.930150 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 985/1949] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:01.941475 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86ca1a-a8a6-4fc6-ba7e-8bae23f82b03 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:01.943279 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-bf86ca1a-a8a6-4fc6-ba7e-8bae23f82b03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:01.950455 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:02.014088 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86ca1a-a8a6-4fc6-ba7e-8bae23f82b03 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:02.023328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d006e40-662f-4f68-a755-f186dad5b698 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:02.023328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d006e40-662f-4f68-a755-f186dad5b698 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:02.023328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d006e40-662f-4f68-a755-f186dad5b698 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:02.023328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d006e40-662f-4f68-a755-f186dad5b698 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:02.029493 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d006e40-662f-4f68-a755-f186dad5b698 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:02.031088 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d006e40-662f-4f68-a755-f186dad5b698 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:02.040902 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 965/1950] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:02.065087 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b73d30-8516-4311-940e-128910781c99 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:02.130936 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:02.150363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b73d30-8516-4311-940e-128910781c99 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:02.159345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9726672-9597-4c87-8436-92fbf0508538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:02.159345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9726672-9597-4c87-8436-92fbf0508538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:02.159345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9726672-9597-4c87-8436-92fbf0508538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:02.159345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9726672-9597-4c87-8436-92fbf0508538 admin admin] PATH_INFO: `/v3/users/e075cd751e554b64baae5cf69eb1e344` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:02.159956 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9726672-9597-4c87-8436-92fbf0508538 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e075cd751e554b64baae5cf69eb1e344)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:02.160198 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9726672-9597-4c87-8436-92fbf0508538 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:02.362871 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-c9726672-9597-4c87-8436-92fbf0508538 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e075cd751e554b64baae5cf69eb1e344'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:02.369893 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-c9726672-9597-4c87-8436-92fbf0508538 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e075cd751e554b64baae5cf69eb1e344'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:02.370692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-c9726672-9597-4c87-8436-92fbf0508538 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e075cd751e554b64baae5cf69eb1e344'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:02.371351 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-c9726672-9597-4c87-8436-92fbf0508538 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e075cd751e554b64baae5cf69eb1e344'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:02.384364 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-c9726672-9597-4c87-8436-92fbf0508538 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e075cd751e554b64baae5cf69eb1e344'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:02.463771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 986/1951] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:46:02 2022] DELETE /identity/v3/users/e075cd751e554b64baae5cf69eb1e344 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:02.468127 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d021c0-6df8-4c5b-ac2c-f11834c3201f None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:02.468415 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d021c0-6df8-4c5b-ac2c-f11834c3201f None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:02.468691 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d021c0-6df8-4c5b-ac2c-f11834c3201f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:02.486214 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-91d021c0-6df8-4c5b-ac2c-f11834c3201f None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:02.487453 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-91d021c0-6df8-4c5b-ac2c-f11834c3201f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:02.546071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 966/1952] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:02.556288 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9726672-9597-4c87-8436-92fbf0508538 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:02.557703 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c9726672-9597-4c87-8436-92fbf0508538 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:02.614675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9726672-9597-4c87-8436-92fbf0508538 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:02.618483 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d1d8b-b018-45bd-8a6b-ff7208960908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:02.619121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d1d8b-b018-45bd-8a6b-ff7208960908 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:02.619405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d1d8b-b018-45bd-8a6b-ff7208960908 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:02.619682 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d1d8b-b018-45bd-8a6b-ff7208960908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:02.629350 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d1d8b-b018-45bd-8a6b-ff7208960908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:02.630851 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d1d8b-b018-45bd-8a6b-ff7208960908 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:02.640423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 987/1953] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:03.039584 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d021c0-6df8-4c5b-ac2c-f11834c3201f None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:03.043599 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-91d021c0-6df8-4c5b-ac2c-f11834c3201f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:03.056420 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d021c0-6df8-4c5b-ac2c-f11834c3201f None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:03.061351 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910a107-b678-4bc7-9765-fb3e85633176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:03.062106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910a107-b678-4bc7-9765-fb3e85633176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:03.062410 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910a107-b678-4bc7-9765-fb3e85633176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:03.062706 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910a107-b678-4bc7-9765-fb3e85633176 admin admin] PATH_INFO: `/v3/projects/031c5509bf994c0b889910d29051ebec` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:03.063907 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910a107-b678-4bc7-9765-fb3e85633176 admin admin] RBAC: Authorizing `identity:delete_project(project_id=031c5509bf994c0b889910d29051ebec)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:03.066155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910a107-b678-4bc7-9765-fb3e85633176 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:03.180766 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-0910a107-b678-4bc7-9765-fb3e85633176 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '031c5509bf994c0b889910d29051ebec', 'request_id': 'req-0910a107-b678-4bc7-9765-fb3e85633176'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:03.258693 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-0910a107-b678-4bc7-9765-fb3e85633176 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '031c5509bf994c0b889910d29051ebec', 'request_id': 'req-0910a107-b678-4bc7-9765-fb3e85633176'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:03.265851 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-0910a107-b678-4bc7-9765-fb3e85633176 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '031c5509bf994c0b889910d29051ebec', 'request_id': 'req-0910a107-b678-4bc7-9765-fb3e85633176'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:03.277632 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-0910a107-b678-4bc7-9765-fb3e85633176 admin admin] The token cache is being invalidate because project 031c5509bf994c0b889910d29051ebec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:46:03.277927 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-0910a107-b678-4bc7-9765-fb3e85633176 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:03.278647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-0910a107-b678-4bc7-9765-fb3e85633176 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:03.280213 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 967/1954] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:46:03 2022] DELETE /identity/v3/projects/031c5509bf994c0b889910d29051ebec => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:03.286889 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b67d1-9b78-4e49-a005-00406bfb0bbd None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:03.286889 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b67d1-9b78-4e49-a005-00406bfb0bbd None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:03.286889 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b67d1-9b78-4e49-a005-00406bfb0bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:03.303640 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-f81b67d1-9b78-4e49-a005-00406bfb0bbd None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:03.305782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-f81b67d1-9b78-4e49-a005-00406bfb0bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:03.376301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 988/1955] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:03.384545 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ae8b8-bf8f-45de-864d-283f12449300 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:03.384835 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ae8b8-bf8f-45de-864d-283f12449300 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:03.385105 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ae8b8-bf8f-45de-864d-283f12449300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:03.403987 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-679ae8b8-bf8f-45de-864d-283f12449300 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:03.406359 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-679ae8b8-bf8f-45de-864d-283f12449300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:03.413493 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 968/1956] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:03.417866 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b67d1-9b78-4e49-a005-00406bfb0bbd None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:03.423727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:03.427574 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b67d1-9b78-4e49-a005-00406bfb0bbd None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:03.433346 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48819813-8816-407f-adfa-f1f930e651cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:03.434100 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48819813-8816-407f-adfa-f1f930e651cc admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:03.437471 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48819813-8816-407f-adfa-f1f930e651cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:03.437471 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48819813-8816-407f-adfa-f1f930e651cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:03.437471 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48819813-8816-407f-adfa-f1f930e651cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:46:03.437471 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48819813-8816-407f-adfa-f1f930e651cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:03.437471 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48819813-8816-407f-adfa-f1f930e651cc admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:03.451481 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 989/1957] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:46:03.455877 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ae8b8-bf8f-45de-864d-283f12449300 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:03.466446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ae8b8-bf8f-45de-864d-283f12449300 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:03.471515 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c7057-9fc0-40ef-9589-f0f311ef3bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:03.472249 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c7057-9fc0-40ef-9589-f0f311ef3bf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:03.472603 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c7057-9fc0-40ef-9589-f0f311ef3bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:03.472949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c7057-9fc0-40ef-9589-f0f311ef3bf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:03.473647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c7057-9fc0-40ef-9589-f0f311ef3bf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:03.475645 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c7057-9fc0-40ef-9589-f0f311ef3bf9 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:03.563222 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 969/1958] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:46:03 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:46:03.567333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48819813-8816-407f-adfa-f1f930e651cc admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:03.579465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48819813-8816-407f-adfa-f1f930e651cc admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:03.636214 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e180ecf-6c57-4b91-8266-fd50f94301b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:03.636877 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e180ecf-6c57-4b91-8266-fd50f94301b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:03.637168 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e180ecf-6c57-4b91-8266-fd50f94301b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:03.637446 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e180ecf-6c57-4b91-8266-fd50f94301b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:03.638126 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e180ecf-6c57-4b91-8266-fd50f94301b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:03.640092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e180ecf-6c57-4b91-8266-fd50f94301b9 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:03.729762 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 990/1959] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:46:03 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:03.731082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c7057-9fc0-40ef-9589-f0f311ef3bf9 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:03.741273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:03.743514 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c7057-9fc0-40ef-9589-f0f311ef3bf9 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:03.748172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc5f98-c44c-474c-bc30-f4c8ebde6fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:03.748851 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc5f98-c44c-474c-bc30-f4c8ebde6fcb admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:03.749117 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc5f98-c44c-474c-bc30-f4c8ebde6fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:03.749393 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc5f98-c44c-474c-bc30-f4c8ebde6fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:03.750500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc5f98-c44c-474c-bc30-f4c8ebde6fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:03.751741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc5f98-c44c-474c-bc30-f4c8ebde6fcb admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:03.763423 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 970/1960] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:03.772265 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e180ecf-6c57-4b91-8266-fd50f94301b9 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:03.783356 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e180ecf-6c57-4b91-8266-fd50f94301b9 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:03.787840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7431a-96c2-4ef2-a60c-f77666ba300a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:03.788700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7431a-96c2-4ef2-a60c-f77666ba300a admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:03.788811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7431a-96c2-4ef2-a60c-f77666ba300a admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:03.789055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7431a-96c2-4ef2-a60c-f77666ba300a admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:03.789748 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7431a-96c2-4ef2-a60c-f77666ba300a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:03.791365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7431a-96c2-4ef2-a60c-f77666ba300a admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:03.801362 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 991/1961] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:03.805607 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc5f98-c44c-474c-bc30-f4c8ebde6fcb admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:03.815397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc5f98-c44c-474c-bc30-f4c8ebde6fcb admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:03.821070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a518f-a845-4221-918f-f7c20d2c701d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:03.821873 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a518f-a845-4221-918f-f7c20d2c701d admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:03.822162 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a518f-a845-4221-918f-f7c20d2c701d admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:03.822482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a518f-a845-4221-918f-f7c20d2c701d admin admin] PATH_INFO: `/v3/projects/d22f0e7a3932487b8bac214847b61a9a/users/cb695cd9c6dd45f29176d6507c9f5458/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:03.837184 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a518f-a845-4221-918f-f7c20d2c701d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d22f0e7a3932487b8bac214847b61a9a, user_id=cb695cd9c6dd45f29176d6507c9f5458, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:03.839882 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a518f-a845-4221-918f-f7c20d2c701d admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:03.917341 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 971/1962] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:03 2022] PUT /identity/v3/projects/d22f0e7a3932487b8bac214847b61a9a/users/cb695cd9c6dd45f29176d6507c9f5458/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:03.927785 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26abcd-d292-4c64-8b0b-1b367ca4d699 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:03.928300 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26abcd-d292-4c64-8b0b-1b367ca4d699 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:03.928878 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26abcd-d292-4c64-8b0b-1b367ca4d699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:03.957956 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-8d26abcd-d292-4c64-8b0b-1b367ca4d699 None None] MFA Rules not processed for user `cb695cd9c6dd45f29176d6507c9f5458`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:03.959196 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-8d26abcd-d292-4c64-8b0b-1b367ca4d699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:04.023491 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 992/1963] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:04.034734 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a518f-a845-4221-918f-f7c20d2c701d admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:04.036670 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-ca8a518f-a845-4221-918f-f7c20d2c701d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:04.094307 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a518f-a845-4221-918f-f7c20d2c701d admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:04.097934 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff8edc-4c9f-46d3-bbda-6d5e0bd35fd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:04.098599 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff8edc-4c9f-46d3-bbda-6d5e0bd35fd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:04.098878 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff8edc-4c9f-46d3-bbda-6d5e0bd35fd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:04.099150 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff8edc-4c9f-46d3-bbda-6d5e0bd35fd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:04.108115 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff8edc-4c9f-46d3-bbda-6d5e0bd35fd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:04.109511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff8edc-4c9f-46d3-bbda-6d5e0bd35fd3 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:04.119355 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 972/1964] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:04.143331 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26abcd-d292-4c64-8b0b-1b367ca4d699 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:04.208583 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26abcd-d292-4c64-8b0b-1b367ca4d699 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:04.214758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c086f41-4f91-4348-b9b8-b854de4c356d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:04.216226 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c086f41-4f91-4348-b9b8-b854de4c356d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:04.216731 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c086f41-4f91-4348-b9b8-b854de4c356d admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:04.217217 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c086f41-4f91-4348-b9b8-b854de4c356d admin admin] PATH_INFO: `/v3/users/cb695cd9c6dd45f29176d6507c9f5458` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:04.219005 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c086f41-4f91-4348-b9b8-b854de4c356d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb695cd9c6dd45f29176d6507c9f5458)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:04.220929 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c086f41-4f91-4348-b9b8-b854de4c356d admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:04.222956 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:04.434806 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-5c086f41-4f91-4348-b9b8-b854de4c356d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb695cd9c6dd45f29176d6507c9f5458'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:04.442245 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-5c086f41-4f91-4348-b9b8-b854de4c356d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb695cd9c6dd45f29176d6507c9f5458'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:04.443037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-5c086f41-4f91-4348-b9b8-b854de4c356d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb695cd9c6dd45f29176d6507c9f5458'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:04.443684 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-5c086f41-4f91-4348-b9b8-b854de4c356d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb695cd9c6dd45f29176d6507c9f5458'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:04.457189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-5c086f41-4f91-4348-b9b8-b854de4c356d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb695cd9c6dd45f29176d6507c9f5458'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:04.531717 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 993/1965] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:46:04 2022] DELETE /identity/v3/users/cb695cd9c6dd45f29176d6507c9f5458 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:04.536338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce8021-f63b-4042-8334-19b0724b9609 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:04.536664 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce8021-f63b-4042-8334-19b0724b9609 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:04.536941 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce8021-f63b-4042-8334-19b0724b9609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:04.555685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-b7ce8021-f63b-4042-8334-19b0724b9609 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:04.559252 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-b7ce8021-f63b-4042-8334-19b0724b9609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:04.632033 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 973/1966] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:04.641522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c086f41-4f91-4348-b9b8-b854de4c356d admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:04.642955 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-5c086f41-4f91-4348-b9b8-b854de4c356d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:04.703412 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c086f41-4f91-4348-b9b8-b854de4c356d admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:04.707218 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8821b77-9b12-4db6-a890-357652493967 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:04.707866 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8821b77-9b12-4db6-a890-357652493967 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:04.708146 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8821b77-9b12-4db6-a890-357652493967 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:04.708418 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8821b77-9b12-4db6-a890-357652493967 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:04.718046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8821b77-9b12-4db6-a890-357652493967 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:04.719533 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8821b77-9b12-4db6-a890-357652493967 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:04.732587 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 994/1967] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:05.209392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce8021-f63b-4042-8334-19b0724b9609 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:05.212079 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-b7ce8021-f63b-4042-8334-19b0724b9609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:05.224704 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce8021-f63b-4042-8334-19b0724b9609 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:05.239989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa728a6-6214-4212-b32f-99df09ec51b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:05.239989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa728a6-6214-4212-b32f-99df09ec51b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:05.239989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa728a6-6214-4212-b32f-99df09ec51b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:05.239989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa728a6-6214-4212-b32f-99df09ec51b5 admin admin] PATH_INFO: `/v3/projects/d22f0e7a3932487b8bac214847b61a9a` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:05.240258 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa728a6-6214-4212-b32f-99df09ec51b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d22f0e7a3932487b8bac214847b61a9a)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:05.240258 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa728a6-6214-4212-b32f-99df09ec51b5 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:05.381523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2aa728a6-6214-4212-b32f-99df09ec51b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd22f0e7a3932487b8bac214847b61a9a', 'request_id': 'req-2aa728a6-6214-4212-b32f-99df09ec51b5'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:05.494500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2aa728a6-6214-4212-b32f-99df09ec51b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd22f0e7a3932487b8bac214847b61a9a', 'request_id': 'req-2aa728a6-6214-4212-b32f-99df09ec51b5'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:05.501765 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2aa728a6-6214-4212-b32f-99df09ec51b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd22f0e7a3932487b8bac214847b61a9a', 'request_id': 'req-2aa728a6-6214-4212-b32f-99df09ec51b5'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:05.506114 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:05.514402 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2aa728a6-6214-4212-b32f-99df09ec51b5 admin admin] The token cache is being invalidate because project d22f0e7a3932487b8bac214847b61a9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:46:05.514708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2aa728a6-6214-4212-b32f-99df09ec51b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:05.515454 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2aa728a6-6214-4212-b32f-99df09ec51b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:05.517454 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 974/1968] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:46:05 2022] DELETE /identity/v3/projects/d22f0e7a3932487b8bac214847b61a9a => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:05.522841 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69f112-cc9e-43a8-8222-03513b75cb6b None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:05.523409 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69f112-cc9e-43a8-8222-03513b75cb6b None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:05.523809 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69f112-cc9e-43a8-8222-03513b75cb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:05.544296 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-3a69f112-cc9e-43a8-8222-03513b75cb6b None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:05.545508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-3a69f112-cc9e-43a8-8222-03513b75cb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:05.615080 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 995/1969] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:05.623597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8b44d-185c-42a9-8a5a-a88605740ad2 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:05.623891 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8b44d-185c-42a9-8a5a-a88605740ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:05.624189 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8b44d-185c-42a9-8a5a-a88605740ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:05.645238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-30d8b44d-185c-42a9-8a5a-a88605740ad2 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:05.646734 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-30d8b44d-185c-42a9-8a5a-a88605740ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:05.658454 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 975/1970] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:05.660462 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69f112-cc9e-43a8-8222-03513b75cb6b None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:05.670755 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69f112-cc9e-43a8-8222-03513b75cb6b None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:05.675427 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd57d1-6e2e-43fd-9d26-e410b04ef998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:05.676082 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd57d1-6e2e-43fd-9d26-e410b04ef998 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:05.676365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd57d1-6e2e-43fd-9d26-e410b04ef998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:05.676643 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd57d1-6e2e-43fd-9d26-e410b04ef998 admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:05.677375 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd57d1-6e2e-43fd-9d26-e410b04ef998 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:46:05.677682 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd57d1-6e2e-43fd-9d26-e410b04ef998 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:05.679975 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd57d1-6e2e-43fd-9d26-e410b04ef998 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:05.692686 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 996/1971] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:46:05.697802 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d8b44d-185c-42a9-8a5a-a88605740ad2 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:05.707515 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d8b44d-185c-42a9-8a5a-a88605740ad2 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:05.712679 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60e95f-512e-455d-afc5-024bfc2e6726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:05.713481 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60e95f-512e-455d-afc5-024bfc2e6726 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:05.713923 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60e95f-512e-455d-afc5-024bfc2e6726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:05.714276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60e95f-512e-455d-afc5-024bfc2e6726 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:05.715082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60e95f-512e-455d-afc5-024bfc2e6726 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:05.717036 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60e95f-512e-455d-afc5-024bfc2e6726 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:05.821819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 976/1972] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:46:05 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:05.822982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd57d1-6e2e-43fd-9d26-e410b04ef998 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:05.833279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd57d1-6e2e-43fd-9d26-e410b04ef998 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:05.838653 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:05.898516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3e85d-6359-43c8-bfde-5e396712743e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:05.899272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3e85d-6359-43c8-bfde-5e396712743e admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:05.899737 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3e85d-6359-43c8-bfde-5e396712743e admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:05.900014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a3e85d-6359-43c8-bfde-5e396712743e admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:05.900791 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3e85d-6359-43c8-bfde-5e396712743e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:05.903009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a3e85d-6359-43c8-bfde-5e396712743e admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:05.985899 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 997/1973] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:46:05 2022] POST /identity/v3/users => generated 287 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:05.990285 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60e95f-512e-455d-afc5-024bfc2e6726 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:06.001390 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60e95f-512e-455d-afc5-024bfc2e6726 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:06.006215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb4e8e-923e-474e-a7e0-6f0afec945cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:06.007839 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb4e8e-923e-474e-a7e0-6f0afec945cf admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:06.008130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb4e8e-923e-474e-a7e0-6f0afec945cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:06.008411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb4e8e-923e-474e-a7e0-6f0afec945cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:06.009142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb4e8e-923e-474e-a7e0-6f0afec945cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:06.010785 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb4e8e-923e-474e-a7e0-6f0afec945cf admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:06.021366 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 977/1974] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:06.027195 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3e85d-6359-43c8-bfde-5e396712743e admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:06.037040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a3e85d-6359-43c8-bfde-5e396712743e admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:06.042079 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795a245-8946-4b97-8186-e6e252e241e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:06.042759 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795a245-8946-4b97-8186-e6e252e241e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:06.043045 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795a245-8946-4b97-8186-e6e252e241e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:06.043321 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795a245-8946-4b97-8186-e6e252e241e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:06.044064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795a245-8946-4b97-8186-e6e252e241e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:06.045631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795a245-8946-4b97-8186-e6e252e241e7 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:06.063470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 998/1975] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:06.066283 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb4e8e-923e-474e-a7e0-6f0afec945cf admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:06.076922 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb4e8e-923e-474e-a7e0-6f0afec945cf admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:06.081591 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbd4f06-28f3-4c03-ae5f-bc4b50ffde32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:06.082498 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd4f06-28f3-4c03-ae5f-bc4b50ffde32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:06.082777 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd4f06-28f3-4c03-ae5f-bc4b50ffde32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:06.083050 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd4f06-28f3-4c03-ae5f-bc4b50ffde32 admin admin] PATH_INFO: `/v3/projects/0343ff062aba4b67b28b2ecf2493d2f1/users/eb023516388a4664a4dedb936020c8ed/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:06.099440 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbd4f06-28f3-4c03-ae5f-bc4b50ffde32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0343ff062aba4b67b28b2ecf2493d2f1, user_id=eb023516388a4664a4dedb936020c8ed, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:06.102448 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbd4f06-28f3-4c03-ae5f-bc4b50ffde32 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:06.162870 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 978/1976] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:06 2022] PUT /identity/v3/projects/0343ff062aba4b67b28b2ecf2493d2f1/users/eb023516388a4664a4dedb936020c8ed/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:06.174323 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7555a1-7d3b-4594-baab-374c1238e11d None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:06.174518 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7555a1-7d3b-4594-baab-374c1238e11d None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:06.174811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7555a1-7d3b-4594-baab-374c1238e11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:06.204423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-9e7555a1-7d3b-4594-baab-374c1238e11d None None] MFA Rules not processed for user `eb023516388a4664a4dedb936020c8ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:06.205641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-9e7555a1-7d3b-4594-baab-374c1238e11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:06.272773 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 999/1977] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:06.283151 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbd4f06-28f3-4c03-ae5f-bc4b50ffde32 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:06.284967 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-4cbd4f06-28f3-4c03-ae5f-bc4b50ffde32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:06.347145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbd4f06-28f3-4c03-ae5f-bc4b50ffde32 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:06.351096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9fba2-52f2-4df0-a817-cc711b4ce9ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:06.351778 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9fba2-52f2-4df0-a817-cc711b4ce9ed service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:06.352061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9fba2-52f2-4df0-a817-cc711b4ce9ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:06.352332 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9fba2-52f2-4df0-a817-cc711b4ce9ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:06.361998 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9fba2-52f2-4df0-a817-cc711b4ce9ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:06.363608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9fba2-52f2-4df0-a817-cc711b4ce9ed service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:06.373781 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 979/1978] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:13.098476 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7555a1-7d3b-4594-baab-374c1238e11d None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:13.104223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-9e7555a1-7d3b-4594-baab-374c1238e11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:13.167800 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7555a1-7d3b-4594-baab-374c1238e11d None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:13.172209 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e63dd-3fed-41fb-9f65-2847f26a44ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:13.173115 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e63dd-3fed-41fb-9f65-2847f26a44ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:13.173396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e63dd-3fed-41fb-9f65-2847f26a44ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:13.173666 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e63dd-3fed-41fb-9f65-2847f26a44ec admin admin] PATH_INFO: `/v3/users/c78783feb78f42ac92a9766cb4718999` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:13.174865 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e63dd-3fed-41fb-9f65-2847f26a44ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=c78783feb78f42ac92a9766cb4718999)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:13.176774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e63dd-3fed-41fb-9f65-2847f26a44ec admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:13.202173 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:13.415155 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-ae2e63dd-3fed-41fb-9f65-2847f26a44ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c78783feb78f42ac92a9766cb4718999'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:13.421977 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-ae2e63dd-3fed-41fb-9f65-2847f26a44ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c78783feb78f42ac92a9766cb4718999'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:13.422680 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-ae2e63dd-3fed-41fb-9f65-2847f26a44ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c78783feb78f42ac92a9766cb4718999'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:13.423289 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-ae2e63dd-3fed-41fb-9f65-2847f26a44ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c78783feb78f42ac92a9766cb4718999'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:13.435907 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-ae2e63dd-3fed-41fb-9f65-2847f26a44ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c78783feb78f42ac92a9766cb4718999'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:13.510773 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1000/1979] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:46:13 2022] DELETE /identity/v3/users/c78783feb78f42ac92a9766cb4718999 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:13.515405 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc035d-f6c4-4c81-bd96-1cd806242b7d None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:13.515886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc035d-f6c4-4c81-bd96-1cd806242b7d None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:13.515975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc035d-f6c4-4c81-bd96-1cd806242b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:13.534044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-c5dc035d-f6c4-4c81-bd96-1cd806242b7d None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:13.535353 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-c5dc035d-f6c4-4c81-bd96-1cd806242b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:13.602591 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 980/1980] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:13.609676 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e63dd-3fed-41fb-9f65-2847f26a44ec admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:13.613800 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ae2e63dd-3fed-41fb-9f65-2847f26a44ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:13.669921 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e63dd-3fed-41fb-9f65-2847f26a44ec admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:13.673643 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3057dfa-ddc6-4e48-8a05-bc0d1083d05e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:13.674285 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3057dfa-ddc6-4e48-8a05-bc0d1083d05e service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:13.674602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3057dfa-ddc6-4e48-8a05-bc0d1083d05e service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:13.674909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3057dfa-ddc6-4e48-8a05-bc0d1083d05e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:13.684179 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3057dfa-ddc6-4e48-8a05-bc0d1083d05e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:13.685602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3057dfa-ddc6-4e48-8a05-bc0d1083d05e service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:13.696288 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1001/1981] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:14.063274 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc035d-f6c4-4c81-bd96-1cd806242b7d None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:14.064760 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-c5dc035d-f6c4-4c81-bd96-1cd806242b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:14.077461 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc035d-f6c4-4c81-bd96-1cd806242b7d None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:14.082181 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c2c70-b2c3-45e5-8e96-6c191c35364a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:14.083890 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c2c70-b2c3-45e5-8e96-6c191c35364a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:14.083890 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c2c70-b2c3-45e5-8e96-6c191c35364a admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:14.084223 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c2c70-b2c3-45e5-8e96-6c191c35364a admin admin] PATH_INFO: `/v3/users/001c6fdcc55c421e82a6e3f5c1a36fdf` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:14.086187 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c2c70-b2c3-45e5-8e96-6c191c35364a admin admin] RBAC: Authorizing `identity:delete_user(user_id=001c6fdcc55c421e82a6e3f5c1a36fdf)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:14.087981 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c2c70-b2c3-45e5-8e96-6c191c35364a admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:14.193969 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:14.204594 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3057dfa-ddc6-4e48-8a05-bc0d1083d05e service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:14.206091 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-a3057dfa-ddc6-4e48-8a05-bc0d1083d05e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:14.241598 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3057dfa-ddc6-4e48-8a05-bc0d1083d05e service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:14.247112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b33a59-82d3-4c42-b211-9bb5b7280192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:14.248255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b33a59-82d3-4c42-b211-9bb5b7280192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:14.248642 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b33a59-82d3-4c42-b211-9bb5b7280192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:14.249049 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b33a59-82d3-4c42-b211-9bb5b7280192 admin admin] PATH_INFO: `/v3/projects/34fac855cf8146ec8a8553d9c73cc648` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:14.250470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b33a59-82d3-4c42-b211-9bb5b7280192 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34fac855cf8146ec8a8553d9c73cc648)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:14.254570 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b33a59-82d3-4c42-b211-9bb5b7280192 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:14.292042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-cf9c2c70-b2c3-45e5-8e96-6c191c35364a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '001c6fdcc55c421e82a6e3f5c1a36fdf'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:14.299318 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-cf9c2c70-b2c3-45e5-8e96-6c191c35364a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '001c6fdcc55c421e82a6e3f5c1a36fdf'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:14.300025 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-cf9c2c70-b2c3-45e5-8e96-6c191c35364a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '001c6fdcc55c421e82a6e3f5c1a36fdf'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:14.314043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-cf9c2c70-b2c3-45e5-8e96-6c191c35364a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '001c6fdcc55c421e82a6e3f5c1a36fdf'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:14.437999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-c4b33a59-82d3-4c42-b211-9bb5b7280192 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34fac855cf8146ec8a8553d9c73cc648', 'request_id': 'req-c4b33a59-82d3-4c42-b211-9bb5b7280192'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:14.448690 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-cf9c2c70-b2c3-45e5-8e96-6c191c35364a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '001c6fdcc55c421e82a6e3f5c1a36fdf'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:14.457338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 981/1982] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:46:14 2022] DELETE /identity/v3/users/001c6fdcc55c421e82a6e3f5c1a36fdf => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:14.461840 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967eb795-356c-4ecc-907c-7cfb2cf701e0 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:14.462132 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967eb795-356c-4ecc-907c-7cfb2cf701e0 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:14.462615 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967eb795-356c-4ecc-907c-7cfb2cf701e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:14.480485 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-967eb795-356c-4ecc-907c-7cfb2cf701e0 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:14.481983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-967eb795-356c-4ecc-907c-7cfb2cf701e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:14.522975 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-c4b33a59-82d3-4c42-b211-9bb5b7280192 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34fac855cf8146ec8a8553d9c73cc648', 'request_id': 'req-c4b33a59-82d3-4c42-b211-9bb5b7280192'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:14.555206 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 982/1983] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:46:14.555861 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-c4b33a59-82d3-4c42-b211-9bb5b7280192 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34fac855cf8146ec8a8553d9c73cc648', 'request_id': 'req-c4b33a59-82d3-4c42-b211-9bb5b7280192'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:14.556873 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967eb795-356c-4ecc-907c-7cfb2cf701e0 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:14.558478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-967eb795-356c-4ecc-907c-7cfb2cf701e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:14.588978 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-c4b33a59-82d3-4c42-b211-9bb5b7280192 admin admin] The token cache is being invalidate because project 34fac855cf8146ec8a8553d9c73cc648 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:46:14.589281 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-c4b33a59-82d3-4c42-b211-9bb5b7280192 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:14.590109 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-c4b33a59-82d3-4c42-b211-9bb5b7280192 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:14.592070 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1002/1984] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:46:14 2022] DELETE /identity/v3/projects/34fac855cf8146ec8a8553d9c73cc648 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:14.596355 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b33a59-82d3-4c42-b211-9bb5b7280192 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:14.597926 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c4b33a59-82d3-4c42-b211-9bb5b7280192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:14.627785 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967eb795-356c-4ecc-907c-7cfb2cf701e0 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:14.633603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:14.641541 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145708ba-0571-4308-a548-a8c3e6af6f3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:14.642210 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145708ba-0571-4308-a548-a8c3e6af6f3b service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:14.642530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145708ba-0571-4308-a548-a8c3e6af6f3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:14.642811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145708ba-0571-4308-a548-a8c3e6af6f3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:14.644676 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-145708ba-0571-4308-a548-a8c3e6af6f3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:14.658914 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145708ba-0571-4308-a548-a8c3e6af6f3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:14.660606 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145708ba-0571-4308-a548-a8c3e6af6f3b service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:14.675621 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 983/1985] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:14.691451 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b33a59-82d3-4c42-b211-9bb5b7280192 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:14.697554 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a43865-7ccc-4dde-a1d2-6d9a5971e0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:14.698838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a43865-7ccc-4dde-a1d2-6d9a5971e0ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:14.699124 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a43865-7ccc-4dde-a1d2-6d9a5971e0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:14.699403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a43865-7ccc-4dde-a1d2-6d9a5971e0ec admin admin] PATH_INFO: `/v3/users/8ec088c3a48f42b1af55d10993d676ee` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:14.700629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a43865-7ccc-4dde-a1d2-6d9a5971e0ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ec088c3a48f42b1af55d10993d676ee)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:14.704112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a43865-7ccc-4dde-a1d2-6d9a5971e0ec admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:14.929401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-31a43865-7ccc-4dde-a1d2-6d9a5971e0ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ec088c3a48f42b1af55d10993d676ee'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:14.937006 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-31a43865-7ccc-4dde-a1d2-6d9a5971e0ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ec088c3a48f42b1af55d10993d676ee'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:14.937762 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-31a43865-7ccc-4dde-a1d2-6d9a5971e0ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ec088c3a48f42b1af55d10993d676ee'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:14.938512 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-31a43865-7ccc-4dde-a1d2-6d9a5971e0ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ec088c3a48f42b1af55d10993d676ee'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:14.960596 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-31a43865-7ccc-4dde-a1d2-6d9a5971e0ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ec088c3a48f42b1af55d10993d676ee'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:15.062179 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1003/1986] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:46:14 2022] DELETE /identity/v3/users/8ec088c3a48f42b1af55d10993d676ee => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:15.573701 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145708ba-0571-4308-a548-a8c3e6af6f3b service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:15.575196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-145708ba-0571-4308-a548-a8c3e6af6f3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:15.599585 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145708ba-0571-4308-a548-a8c3e6af6f3b service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:15.658667 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:15.672863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576cf4b8-bb96-4a13-bfbf-0702aa800188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:15.673816 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576cf4b8-bb96-4a13-bfbf-0702aa800188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:15.674166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576cf4b8-bb96-4a13-bfbf-0702aa800188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:15.674542 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576cf4b8-bb96-4a13-bfbf-0702aa800188 admin admin] PATH_INFO: `/v3/projects/f7ebbaa0a00b41bd8af1a4401c392193` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:15.675798 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576cf4b8-bb96-4a13-bfbf-0702aa800188 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7ebbaa0a00b41bd8af1a4401c392193)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:15.677811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576cf4b8-bb96-4a13-bfbf-0702aa800188 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:16.009405 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-576cf4b8-bb96-4a13-bfbf-0702aa800188 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7ebbaa0a00b41bd8af1a4401c392193', 'request_id': 'req-576cf4b8-bb96-4a13-bfbf-0702aa800188'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:16.024979 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a43865-7ccc-4dde-a1d2-6d9a5971e0ec admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:16.026584 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-31a43865-7ccc-4dde-a1d2-6d9a5971e0ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:16.043349 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a43865-7ccc-4dde-a1d2-6d9a5971e0ec admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:16.123551 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-576cf4b8-bb96-4a13-bfbf-0702aa800188 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7ebbaa0a00b41bd8af1a4401c392193', 'request_id': 'req-576cf4b8-bb96-4a13-bfbf-0702aa800188'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:16.133048 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-576cf4b8-bb96-4a13-bfbf-0702aa800188 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7ebbaa0a00b41bd8af1a4401c392193', 'request_id': 'req-576cf4b8-bb96-4a13-bfbf-0702aa800188'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:16.151709 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b16a05-0221-4ad0-a5ce-85ebf6a55d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:16.152589 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b16a05-0221-4ad0-a5ce-85ebf6a55d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:16.155268 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b16a05-0221-4ad0-a5ce-85ebf6a55d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:16.156044 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b16a05-0221-4ad0-a5ce-85ebf6a55d6b admin admin] PATH_INFO: `/v3/projects/3b50437225624a7a916a0256179bf9dc` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:16.160330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b16a05-0221-4ad0-a5ce-85ebf6a55d6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b50437225624a7a916a0256179bf9dc)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:16.164633 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-576cf4b8-bb96-4a13-bfbf-0702aa800188 admin admin] The token cache is being invalidate because project f7ebbaa0a00b41bd8af1a4401c392193 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:46:16.164633 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-576cf4b8-bb96-4a13-bfbf-0702aa800188 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:16.166863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-576cf4b8-bb96-4a13-bfbf-0702aa800188 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:16.168081 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b16a05-0221-4ad0-a5ce-85ebf6a55d6b admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:16.170860 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 984/1987] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:46:15 2022] DELETE /identity/v3/projects/f7ebbaa0a00b41bd8af1a4401c392193 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:16.175906 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d98d06-be03-4bca-95b7-32b38b13323c None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:16.176662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d98d06-be03-4bca-95b7-32b38b13323c None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:16.177242 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d98d06-be03-4bca-95b7-32b38b13323c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:16.212951 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-82d98d06-be03-4bca-95b7-32b38b13323c None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:16.214510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-82d98d06-be03-4bca-95b7-32b38b13323c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:16.227187 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 985/1988] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:16.244006 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a638b2f-75ed-4b2b-bcc9-7fbbc289a001 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:16.245072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a638b2f-75ed-4b2b-bcc9-7fbbc289a001 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:16.245072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a638b2f-75ed-4b2b-bcc9-7fbbc289a001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:16.264368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-3a638b2f-75ed-4b2b-bcc9-7fbbc289a001 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:16.265691 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-3a638b2f-75ed-4b2b-bcc9-7fbbc289a001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:16.279590 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 986/1989] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:16.285304 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a638b2f-75ed-4b2b-bcc9-7fbbc289a001 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:16.290835 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a638b2f-75ed-4b2b-bcc9-7fbbc289a001 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:16.296339 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bcdd7-363a-4dc9-942e-532f45385adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:16.297180 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bcdd7-363a-4dc9-942e-532f45385adf admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:16.297619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bcdd7-363a-4dc9-942e-532f45385adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:16.298061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bcdd7-363a-4dc9-942e-532f45385adf admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:16.298996 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bcdd7-363a-4dc9-942e-532f45385adf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:46:16.299553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bcdd7-363a-4dc9-942e-532f45385adf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:16.301708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bcdd7-363a-4dc9-942e-532f45385adf admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:16.321126 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 987/1990] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:46:16.323004 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bcdd7-363a-4dc9-942e-532f45385adf admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:16.333948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bcdd7-363a-4dc9-942e-532f45385adf admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:16.340708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f518ca-75b0-4fdc-b801-b1b1699e8234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:16.345986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f518ca-75b0-4fdc-b801-b1b1699e8234 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:16.345986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f518ca-75b0-4fdc-b801-b1b1699e8234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:16.345986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f518ca-75b0-4fdc-b801-b1b1699e8234 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:16.345986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f518ca-75b0-4fdc-b801-b1b1699e8234 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:16.346661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f518ca-75b0-4fdc-b801-b1b1699e8234 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:16.372644 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:16.377104 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-e1b16a05-0221-4ad0-a5ce-85ebf6a55d6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b50437225624a7a916a0256179bf9dc', 'request_id': 'req-e1b16a05-0221-4ad0-a5ce-85ebf6a55d6b'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:16.484468 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-e1b16a05-0221-4ad0-a5ce-85ebf6a55d6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b50437225624a7a916a0256179bf9dc', 'request_id': 'req-e1b16a05-0221-4ad0-a5ce-85ebf6a55d6b'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:16.498046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-e1b16a05-0221-4ad0-a5ce-85ebf6a55d6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b50437225624a7a916a0256179bf9dc', 'request_id': 'req-e1b16a05-0221-4ad0-a5ce-85ebf6a55d6b'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:16.504071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 988/1991] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:46:16 2022] POST /identity/v3/projects => generated 375 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:46:16.508138 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f518ca-75b0-4fdc-b801-b1b1699e8234 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:16.528425 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-e1b16a05-0221-4ad0-a5ce-85ebf6a55d6b admin admin] The token cache is being invalidate because project 3b50437225624a7a916a0256179bf9dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:46:16.529780 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-e1b16a05-0221-4ad0-a5ce-85ebf6a55d6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:16.530772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-e1b16a05-0221-4ad0-a5ce-85ebf6a55d6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:16.534192 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1004/1992] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:46:16 2022] DELETE /identity/v3/projects/3b50437225624a7a916a0256179bf9dc => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:46:16.541858 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c63ca-93f4-4ba5-9ad4-7c98b6348318 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:16.547097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c63ca-93f4-4ba5-9ad4-7c98b6348318 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:16.547097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c63ca-93f4-4ba5-9ad4-7c98b6348318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:16.573150 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-401c63ca-93f4-4ba5-9ad4-7c98b6348318 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:16.574757 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-401c63ca-93f4-4ba5-9ad4-7c98b6348318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:16.651647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f518ca-75b0-4fdc-b801-b1b1699e8234 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:16.657213 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6846560-953b-4532-962e-c6303a66484c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:16.658129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6846560-953b-4532-962e-c6303a66484c admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:16.658591 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6846560-953b-4532-962e-c6303a66484c admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:16.659027 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6846560-953b-4532-962e-c6303a66484c admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:16.659888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6846560-953b-4532-962e-c6303a66484c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:16.661974 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6846560-953b-4532-962e-c6303a66484c admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:16.684584 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1005/1993] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:16.692539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3392c-85a8-4c51-9069-48e3fe48d2b8 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:16.692824 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3392c-85a8-4c51-9069-48e3fe48d2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:16.693097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3392c-85a8-4c51-9069-48e3fe48d2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:16.715565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-fdc3392c-85a8-4c51-9069-48e3fe48d2b8 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:16.716852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-fdc3392c-85a8-4c51-9069-48e3fe48d2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:16.727280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1006/1994] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:46:16.733065 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3392c-85a8-4c51-9069-48e3fe48d2b8 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:16.743999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3392c-85a8-4c51-9069-48e3fe48d2b8 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:16.749272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258cdb4b-ddca-432d-b878-29ae81c8f894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:16.750794 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cdb4b-ddca-432d-b878-29ae81c8f894 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:16.751280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cdb4b-ddca-432d-b878-29ae81c8f894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:16.751715 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cdb4b-ddca-432d-b878-29ae81c8f894 admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:16.752871 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258cdb4b-ddca-432d-b878-29ae81c8f894 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:46:16.753351 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258cdb4b-ddca-432d-b878-29ae81c8f894 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:16.763996 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 989/1995] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:46:16 2022] POST /identity/v3/users => generated 288 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:46:16.767916 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6846560-953b-4532-962e-c6303a66484c admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:16.769790 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258cdb4b-ddca-432d-b878-29ae81c8f894 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:16.770922 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-a6846560-953b-4532-962e-c6303a66484c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:16.786155 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1007/1996] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:46:16.790962 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6846560-953b-4532-962e-c6303a66484c admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:16.793864 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258cdb4b-ddca-432d-b878-29ae81c8f894 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:16.796282 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68017962-50b0-4fdf-970d-97de04991830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:16.798067 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68017962-50b0-4fdf-970d-97de04991830 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:16.798501 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68017962-50b0-4fdf-970d-97de04991830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:16.798886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68017962-50b0-4fdf-970d-97de04991830 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:16.799776 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68017962-50b0-4fdf-970d-97de04991830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:16.801714 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68017962-50b0-4fdf-970d-97de04991830 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:16.809051 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258cdb4b-ddca-432d-b878-29ae81c8f894 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:16.815466 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 990/1997] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:16 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:16.817547 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94138ced-806e-477b-94fd-968bb318bd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:16.819703 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94138ced-806e-477b-94fd-968bb318bd96 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:16.820219 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94138ced-806e-477b-94fd-968bb318bd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:16.821248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68017962-50b0-4fdf-970d-97de04991830 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:16.821964 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94138ced-806e-477b-94fd-968bb318bd96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:16.824202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94138ced-806e-477b-94fd-968bb318bd96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:16.828173 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94138ced-806e-477b-94fd-968bb318bd96 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:16.836206 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68017962-50b0-4fdf-970d-97de04991830 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:16.843854 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee104764-1e26-4704-9c35-026d328ed172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:16.845051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee104764-1e26-4704-9c35-026d328ed172 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:16.845632 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee104764-1e26-4704-9c35-026d328ed172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:16.846361 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee104764-1e26-4704-9c35-026d328ed172 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:16.847613 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee104764-1e26-4704-9c35-026d328ed172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:16.850046 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee104764-1e26-4704-9c35-026d328ed172 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:16.864924 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 991/1998] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:46:16.870309 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee104764-1e26-4704-9c35-026d328ed172 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:16.882076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee104764-1e26-4704-9c35-026d328ed172 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:16.887989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400058d-b163-4251-8a97-b10e8c961d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:16.892315 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400058d-b163-4251-8a97-b10e8c961d0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:16.892315 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400058d-b163-4251-8a97-b10e8c961d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:16.892315 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400058d-b163-4251-8a97-b10e8c961d0e admin admin] PATH_INFO: `/v3/projects/4a2dbe71f2c6477f8e4cc2d3d0248b19/users/b1869cd5b42b4846859cbd06ef868133/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:16.921391 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400058d-b163-4251-8a97-b10e8c961d0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a2dbe71f2c6477f8e4cc2d3d0248b19, user_id=b1869cd5b42b4846859cbd06ef868133, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:16.922255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400058d-b163-4251-8a97-b10e8c961d0e admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:16.949191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1008/1999] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:46:16 2022] POST /identity/v3/projects => generated 387 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:16.953558 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94138ced-806e-477b-94fd-968bb318bd96 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:17.031964 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94138ced-806e-477b-94fd-968bb318bd96 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:17.033986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:17.040548 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 992/2000] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:16 2022] PUT /identity/v3/projects/4a2dbe71f2c6477f8e4cc2d3d0248b19/users/b1869cd5b42b4846859cbd06ef868133/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:17.055515 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02e060-9694-4f1c-8f5f-7810baaf5adc None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:17.055917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02e060-9694-4f1c-8f5f-7810baaf5adc None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:17.056291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02e060-9694-4f1c-8f5f-7810baaf5adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:17.104196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-2a02e060-9694-4f1c-8f5f-7810baaf5adc None None] MFA Rules not processed for user `b1869cd5b42b4846859cbd06ef868133`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:17.106358 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-2a02e060-9694-4f1c-8f5f-7810baaf5adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:17.154971 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e75da-bdad-4d2b-bf9b-716d6afe613b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:17.157241 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e75da-bdad-4d2b-bf9b-716d6afe613b admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:17.157660 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e75da-bdad-4d2b-bf9b-716d6afe613b admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:17.157950 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e75da-bdad-4d2b-bf9b-716d6afe613b admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:17.159448 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e75da-bdad-4d2b-bf9b-716d6afe613b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:17.169472 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e75da-bdad-4d2b-bf9b-716d6afe613b admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:17.233773 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 993/2001] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:46:17.246960 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02e060-9694-4f1c-8f5f-7810baaf5adc None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:17.258324 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02e060-9694-4f1c-8f5f-7810baaf5adc None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:17.273268 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1009/2002] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:46:16 2022] POST /identity/v3/users => generated 294 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:17.274222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e75da-bdad-4d2b-bf9b-716d6afe613b admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:17.277218 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411f696-0ecd-424b-b4bb-fa41111df36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:17.277892 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411f696-0ecd-424b-b4bb-fa41111df36c admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:17.278173 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411f696-0ecd-424b-b4bb-fa41111df36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:17.278535 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411f696-0ecd-424b-b4bb-fa41111df36c admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:17.279286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411f696-0ecd-424b-b4bb-fa41111df36c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:17.281369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411f696-0ecd-424b-b4bb-fa41111df36c admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:17.299854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e75da-bdad-4d2b-bf9b-716d6afe613b admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:17.309597 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bcd51-d810-4035-8a5a-117e7b2b00fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:17.313036 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bcd51-d810-4035-8a5a-117e7b2b00fb admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:17.313036 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bcd51-d810-4035-8a5a-117e7b2b00fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:17.313036 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bcd51-d810-4035-8a5a-117e7b2b00fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:17.313036 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3bcd51-d810-4035-8a5a-117e7b2b00fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:17.314002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3bcd51-d810-4035-8a5a-117e7b2b00fb admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:17.336424 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1010/2003] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:17.339283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bcd51-d810-4035-8a5a-117e7b2b00fb admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:17.355154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bcd51-d810-4035-8a5a-117e7b2b00fb admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:17.361387 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4819a9f-bcd2-4598-a844-af1ecf3a0fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:17.362146 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4819a9f-bcd2-4598-a844-af1ecf3a0fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:17.362582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4819a9f-bcd2-4598-a844-af1ecf3a0fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:17.364588 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4819a9f-bcd2-4598-a844-af1ecf3a0fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:17.365553 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4819a9f-bcd2-4598-a844-af1ecf3a0fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:17.369542 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4819a9f-bcd2-4598-a844-af1ecf3a0fa3 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:17.383810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1011/2004] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:17.387777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4819a9f-bcd2-4598-a844-af1ecf3a0fa3 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:17.391360 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 994/2005] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:46:17 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:46:17.397267 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411f696-0ecd-424b-b4bb-fa41111df36c admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:17.410619 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4819a9f-bcd2-4598-a844-af1ecf3a0fa3 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:17.499402 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:17.510260 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411f696-0ecd-424b-b4bb-fa41111df36c admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:17.518457 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfe30d-f69c-49b9-80b8-5b5b34474348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:17.519587 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfe30d-f69c-49b9-80b8-5b5b34474348 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:17.520003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfe30d-f69c-49b9-80b8-5b5b34474348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:17.520401 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfe30d-f69c-49b9-80b8-5b5b34474348 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:17.522423 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfe30d-f69c-49b9-80b8-5b5b34474348 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:17.524696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfe30d-f69c-49b9-80b8-5b5b34474348 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:17.533529 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a49e5f-bd3e-40e6-8b6e-ab8653769864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:17.534329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a49e5f-bd3e-40e6-8b6e-ab8653769864 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:17.534643 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a49e5f-bd3e-40e6-8b6e-ab8653769864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:17.534924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a49e5f-bd3e-40e6-8b6e-ab8653769864 admin admin] PATH_INFO: `/v3/projects/6457b8c69ee149ec9399da76a424d877/users/5d8f36b08f4a4c97b9deb9afde57fe7a/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:17.561389 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a49e5f-bd3e-40e6-8b6e-ab8653769864 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6457b8c69ee149ec9399da76a424d877, user_id=5d8f36b08f4a4c97b9deb9afde57fe7a, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:17.564811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a49e5f-bd3e-40e6-8b6e-ab8653769864 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:17.681121 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 995/2006] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:46:17 2022] POST /identity/v3/users => generated 294 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:17.685925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfe30d-f69c-49b9-80b8-5b5b34474348 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:17.694552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1012/2007] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:17 2022] PUT /identity/v3/projects/6457b8c69ee149ec9399da76a424d877/users/5d8f36b08f4a4c97b9deb9afde57fe7a/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:17.700578 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfe30d-f69c-49b9-80b8-5b5b34474348 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:17.707090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b938c2-d90a-43b4-8a77-e6a9d23b8a6e None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:17.707090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b938c2-d90a-43b4-8a77-e6a9d23b8a6e None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:17.707090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b938c2-d90a-43b4-8a77-e6a9d23b8a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:17.778729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-d1b938c2-d90a-43b4-8a77-e6a9d23b8a6e None None] MFA Rules not processed for user `5d8f36b08f4a4c97b9deb9afde57fe7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:17.781732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-d1b938c2-d90a-43b4-8a77-e6a9d23b8a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:17.829451 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f273d-3a7d-4f54-a47c-f7cc5e3a81ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:17.833148 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f273d-3a7d-4f54-a47c-f7cc5e3a81ed admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:17.833615 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f273d-3a7d-4f54-a47c-f7cc5e3a81ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:17.833975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f273d-3a7d-4f54-a47c-f7cc5e3a81ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:17.835383 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f273d-3a7d-4f54-a47c-f7cc5e3a81ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:17.837191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f273d-3a7d-4f54-a47c-f7cc5e3a81ed admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:17.856373 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 996/2008] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:17.861539 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f273d-3a7d-4f54-a47c-f7cc5e3a81ed admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:17.879602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f273d-3a7d-4f54-a47c-f7cc5e3a81ed admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:17.895172 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1013/2009] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:46:17.900609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b938c2-d90a-43b4-8a77-e6a9d23b8a6e None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:17.901844 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8379769-c66e-4e7a-a267-4cb942da535a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:17.904551 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8379769-c66e-4e7a-a267-4cb942da535a admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:17.905890 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8379769-c66e-4e7a-a267-4cb942da535a admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:17.906411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8379769-c66e-4e7a-a267-4cb942da535a admin admin] PATH_INFO: `/v3/projects/f12950d215234ca5889adb775a5eb6bd/users/bb4e46f4b08a489d9aec77bfb2b865b7/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:17.917532 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b938c2-d90a-43b4-8a77-e6a9d23b8a6e None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:17.922990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f59cd0-6e59-4d14-970c-fed4412ae5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:17.923997 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f59cd0-6e59-4d14-970c-fed4412ae5a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:17.924430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f59cd0-6e59-4d14-970c-fed4412ae5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:17.924960 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f59cd0-6e59-4d14-970c-fed4412ae5a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:17.925868 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f59cd0-6e59-4d14-970c-fed4412ae5a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:17.928361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f59cd0-6e59-4d14-970c-fed4412ae5a4 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:17.943312 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:17.952634 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8379769-c66e-4e7a-a267-4cb942da535a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f12950d215234ca5889adb775a5eb6bd, user_id=bb4e46f4b08a489d9aec77bfb2b865b7, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:17.956088 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8379769-c66e-4e7a-a267-4cb942da535a admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:18.064882 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 997/2010] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:17 2022] PUT /identity/v3/projects/f12950d215234ca5889adb775a5eb6bd/users/bb4e46f4b08a489d9aec77bfb2b865b7/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:46:18.066087 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8379769-c66e-4e7a-a267-4cb942da535a admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:18.071149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1014/2011] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:46:17 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:18.074085 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f59cd0-6e59-4d14-970c-fed4412ae5a4 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:18.167628 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8379769-c66e-4e7a-a267-4cb942da535a admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:18.256920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d85dd2-3cc9-4d45-b5a3-e4bbeabc4514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:18.258228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d85dd2-3cc9-4d45-b5a3-e4bbeabc4514 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:18.259161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d85dd2-3cc9-4d45-b5a3-e4bbeabc4514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:18.260271 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d85dd2-3cc9-4d45-b5a3-e4bbeabc4514 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:18.261225 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d85dd2-3cc9-4d45-b5a3-e4bbeabc4514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:18.263904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d85dd2-3cc9-4d45-b5a3-e4bbeabc4514 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:18.282733 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 998/2012] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:18.283857 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d85dd2-3cc9-4d45-b5a3-e4bbeabc4514 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:18.298608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d85dd2-3cc9-4d45-b5a3-e4bbeabc4514 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:18.305023 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ded8e3-9079-48c9-8c98-b7932ef3c332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:18.306340 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ded8e3-9079-48c9-8c98-b7932ef3c332 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:18.306876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ded8e3-9079-48c9-8c98-b7932ef3c332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:18.307820 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ded8e3-9079-48c9-8c98-b7932ef3c332 admin admin] PATH_INFO: `/v3/projects/f12950d215234ca5889adb775a5eb6bd/users/bb4e46f4b08a489d9aec77bfb2b865b7/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:18.310536 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ded8e3-9079-48c9-8c98-b7932ef3c332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f12950d215234ca5889adb775a5eb6bd, user_id=bb4e46f4b08a489d9aec77bfb2b865b7, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:18.313307 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ded8e3-9079-48c9-8c98-b7932ef3c332 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:18.335528 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f59cd0-6e59-4d14-970c-fed4412ae5a4 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:18.342323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 999/2013] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:18 2022] PUT /identity/v3/projects/f12950d215234ca5889adb775a5eb6bd/users/bb4e46f4b08a489d9aec77bfb2b865b7/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:18.353294 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9319bf-18d4-4d37-8e3c-fe78c55937dd None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:18.355395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9319bf-18d4-4d37-8e3c-fe78c55937dd None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:18.355395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9319bf-18d4-4d37-8e3c-fe78c55937dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:18.374216 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:18.402666 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-0f9319bf-18d4-4d37-8e3c-fe78c55937dd None None] MFA Rules not processed for user `bb4e46f4b08a489d9aec77bfb2b865b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:18.404865 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-0f9319bf-18d4-4d37-8e3c-fe78c55937dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:18.457726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:18.521179 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204049f-73a8-47b3-99a5-53a8065e27dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:18.522615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204049f-73a8-47b3-99a5-53a8065e27dd admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:18.522757 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204049f-73a8-47b3-99a5-53a8065e27dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:18.523026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204049f-73a8-47b3-99a5-53a8065e27dd admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:18.523723 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4204049f-73a8-47b3-99a5-53a8065e27dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:18.528025 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4204049f-73a8-47b3-99a5-53a8065e27dd admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:18.574500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1000/2014] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:18.630256 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9319bf-18d4-4d37-8e3c-fe78c55937dd None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:18.630256 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-0f9319bf-18d4-4d37-8e3c-fe78c55937dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:18.644294 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1015/2015] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:46:18 2022] POST /identity/v3/users => generated 293 bytes in 571 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:46:18.648149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204049f-73a8-47b3-99a5-53a8065e27dd admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:18.659439 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204049f-73a8-47b3-99a5-53a8065e27dd admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:18.675209 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06138d5e-d7e3-4eb6-9d0f-60de8b1976b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:18.676000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06138d5e-d7e3-4eb6-9d0f-60de8b1976b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:18.676312 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06138d5e-d7e3-4eb6-9d0f-60de8b1976b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:18.676587 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06138d5e-d7e3-4eb6-9d0f-60de8b1976b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:18.677320 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06138d5e-d7e3-4eb6-9d0f-60de8b1976b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:18.682032 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06138d5e-d7e3-4eb6-9d0f-60de8b1976b2 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:18.694223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1016/2016] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:46:18.698211 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06138d5e-d7e3-4eb6-9d0f-60de8b1976b2 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:18.709954 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06138d5e-d7e3-4eb6-9d0f-60de8b1976b2 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:18.725401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc755aa-3f4a-413f-8e83-ba5972f80c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:18.726351 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc755aa-3f4a-413f-8e83-ba5972f80c20 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:18.726736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc755aa-3f4a-413f-8e83-ba5972f80c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:18.727082 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc755aa-3f4a-413f-8e83-ba5972f80c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:18.727895 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc755aa-3f4a-413f-8e83-ba5972f80c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:18.730163 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc755aa-3f4a-413f-8e83-ba5972f80c20 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:18.735287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9319bf-18d4-4d37-8e3c-fe78c55937dd None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:18.739947 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a298f-6b80-49ec-87e9-f305433a8be8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:18.740265 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a298f-6b80-49ec-87e9-f305433a8be8 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:18.741363 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a298f-6b80-49ec-87e9-f305433a8be8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:18.741798 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a298f-6b80-49ec-87e9-f305433a8be8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:18.751452 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1017/2017] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:46:18.756610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc755aa-3f4a-413f-8e83-ba5972f80c20 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:18.774487 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc755aa-3f4a-413f-8e83-ba5972f80c20 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:18.780988 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65f71e-2fd1-4ca4-bc33-90379159093b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:18.782001 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65f71e-2fd1-4ca4-bc33-90379159093b admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:18.782613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65f71e-2fd1-4ca4-bc33-90379159093b admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:18.782989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65f71e-2fd1-4ca4-bc33-90379159093b admin admin] PATH_INFO: `/v3/projects/b64091c38ed04759a2b6bc51ff0e56cb/users/80d8a557927d45e8a002e44d8bc8f479/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:18.803867 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c65f71e-2fd1-4ca4-bc33-90379159093b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b64091c38ed04759a2b6bc51ff0e56cb, user_id=80d8a557927d45e8a002e44d8bc8f479, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:18.807727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c65f71e-2fd1-4ca4-bc33-90379159093b admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:18.833854 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a298f-6b80-49ec-87e9-f305433a8be8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:18.836576 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a298f-6b80-49ec-87e9-f305433a8be8 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:18.850646 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1001/2018] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:46:18.888199 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1018/2019] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:18 2022] PUT /identity/v3/projects/b64091c38ed04759a2b6bc51ff0e56cb/users/80d8a557927d45e8a002e44d8bc8f479/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:46:18.902611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766bfab-996d-4d4a-93aa-df6923c0aba1 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:18.903321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766bfab-996d-4d4a-93aa-df6923c0aba1 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:18.903751 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766bfab-996d-4d4a-93aa-df6923c0aba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:18.941786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-6766bfab-996d-4d4a-93aa-df6923c0aba1 None None] MFA Rules not processed for user `80d8a557927d45e8a002e44d8bc8f479`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:18.944181 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-6766bfab-996d-4d4a-93aa-df6923c0aba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:19.034774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1002/2020] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:19.039228 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65f71e-2fd1-4ca4-bc33-90379159093b admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:19.095611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:19.112396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65f71e-2fd1-4ca4-bc33-90379159093b admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:19.118336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b3334-873c-4552-90b6-f02cef3018f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:19.119235 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b3334-873c-4552-90b6-f02cef3018f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:19.119522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b3334-873c-4552-90b6-f02cef3018f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:19.120123 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b3334-873c-4552-90b6-f02cef3018f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:19.120567 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b3334-873c-4552-90b6-f02cef3018f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:19.122652 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b3334-873c-4552-90b6-f02cef3018f5 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:19.242559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766bfab-996d-4d4a-93aa-df6923c0aba1 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:19.244906 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1019/2021] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:46:19 2022] POST /identity/v3/projects => generated 385 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:19.248850 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:19.258369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766bfab-996d-4d4a-93aa-df6923c0aba1 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:19.350325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55367412-b366-4440-8b6e-17e1d403f1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:19.351713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55367412-b366-4440-8b6e-17e1d403f1be admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:19.352255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55367412-b366-4440-8b6e-17e1d403f1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:19.352450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55367412-b366-4440-8b6e-17e1d403f1be admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:19.353931 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55367412-b366-4440-8b6e-17e1d403f1be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:19.356132 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55367412-b366-4440-8b6e-17e1d403f1be admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:19.455606 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1003/2022] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:46:19 2022] POST /identity/v3/users => generated 299 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:19.459307 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b3334-873c-4552-90b6-f02cef3018f5 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:19.472313 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b3334-873c-4552-90b6-f02cef3018f5 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:19.480176 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e020d48-842f-423b-9046-db838ccb2e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:19.480883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e020d48-842f-423b-9046-db838ccb2e17 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:19.487568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e020d48-842f-423b-9046-db838ccb2e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:19.487568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e020d48-842f-423b-9046-db838ccb2e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:19.487568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e020d48-842f-423b-9046-db838ccb2e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:19.487568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e020d48-842f-423b-9046-db838ccb2e17 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:19.501784 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1020/2023] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:19.502502 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55367412-b366-4440-8b6e-17e1d403f1be admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:19.518432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55367412-b366-4440-8b6e-17e1d403f1be admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:19.528785 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7793c86-ff0a-4224-b649-4b772112c816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:19.529694 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7793c86-ff0a-4224-b649-4b772112c816 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:19.530011 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7793c86-ff0a-4224-b649-4b772112c816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:19.530291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7793c86-ff0a-4224-b649-4b772112c816 admin admin] PATH_INFO: `/v3/projects/e8d467159fde4b81b8af329991d50528/users/b04ebd2f125e4eae9ba4334c18f36797/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:19.550660 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7793c86-ff0a-4224-b649-4b772112c816 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8d467159fde4b81b8af329991d50528, user_id=b04ebd2f125e4eae9ba4334c18f36797, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:19.554210 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7793c86-ff0a-4224-b649-4b772112c816 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:19.656533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1004/2024] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:19 2022] PUT /identity/v3/projects/e8d467159fde4b81b8af329991d50528/users/b04ebd2f125e4eae9ba4334c18f36797/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:46:19.660216 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e020d48-842f-423b-9046-db838ccb2e17 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:19.671673 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e020d48-842f-423b-9046-db838ccb2e17 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:19.769149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d8b9b-746d-456d-abb1-274a47a6dc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:19.770558 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d8b9b-746d-456d-abb1-274a47a6dc6e admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:19.773634 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d8b9b-746d-456d-abb1-274a47a6dc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:19.773634 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d8b9b-746d-456d-abb1-274a47a6dc6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:19.774053 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1d8b9b-746d-456d-abb1-274a47a6dc6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:19.776386 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1d8b9b-746d-456d-abb1-274a47a6dc6e admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:19.798869 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1021/2025] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:19.800088 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7793c86-ff0a-4224-b649-4b772112c816 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:19.815212 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7793c86-ff0a-4224-b649-4b772112c816 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:19.823163 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b6e06-01db-4cbe-8d48-8b23fe442be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:19.828299 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b6e06-01db-4cbe-8d48-8b23fe442be3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:19.828299 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b6e06-01db-4cbe-8d48-8b23fe442be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:19.828299 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b6e06-01db-4cbe-8d48-8b23fe442be3 admin admin] PATH_INFO: `/v3/projects/e8d467159fde4b81b8af329991d50528/users/b04ebd2f125e4eae9ba4334c18f36797/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:19.828299 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b6e06-01db-4cbe-8d48-8b23fe442be3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8d467159fde4b81b8af329991d50528, user_id=b04ebd2f125e4eae9ba4334c18f36797, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:19.829913 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b6e06-01db-4cbe-8d48-8b23fe442be3 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:19.842895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1005/2026] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:19 2022] PUT /identity/v3/projects/e8d467159fde4b81b8af329991d50528/users/b04ebd2f125e4eae9ba4334c18f36797/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:19.852915 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5e692-2e42-42c7-ae52-6df603ca2627 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:19.855980 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5e692-2e42-42c7-ae52-6df603ca2627 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:19.855980 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5e692-2e42-42c7-ae52-6df603ca2627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:19.885777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-49b5e692-2e42-42c7-ae52-6df603ca2627 None None] MFA Rules not processed for user `b04ebd2f125e4eae9ba4334c18f36797`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:19.887629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-49b5e692-2e42-42c7-ae52-6df603ca2627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:19.989439 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1022/2027] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:20.003306 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b6e06-01db-4cbe-8d48-8b23fe442be3 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:20.062991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b6e06-01db-4cbe-8d48-8b23fe442be3 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:20.066705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df430a-8b58-4f3d-ad11-629f11d15846 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:20.067521 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df430a-8b58-4f3d-ad11-629f11d15846 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:20.067808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df430a-8b58-4f3d-ad11-629f11d15846 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:20.068082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df430a-8b58-4f3d-ad11-629f11d15846 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:20.077693 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df430a-8b58-4f3d-ad11-629f11d15846 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:20.083455 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df430a-8b58-4f3d-ad11-629f11d15846 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:20.091002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1006/2028] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:21.357286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5e692-2e42-42c7-ae52-6df603ca2627 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:21.358809 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-49b5e692-2e42-42c7-ae52-6df603ca2627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:21.372508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5e692-2e42-42c7-ae52-6df603ca2627 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:21.376300 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a800bf-57da-4260-9640-086817d49a06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:21.377058 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a800bf-57da-4260-9640-086817d49a06 service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:21.377454 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a800bf-57da-4260-9640-086817d49a06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:21.377828 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a800bf-57da-4260-9640-086817d49a06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:21.421460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:21.442628 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a800bf-57da-4260-9640-086817d49a06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:21.442628 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a800bf-57da-4260-9640-086817d49a06 service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:21.449705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1023/2029] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:25.700994 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df430a-8b58-4f3d-ad11-629f11d15846 service cinder] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:25.766718 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:25.770913 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df430a-8b58-4f3d-ad11-629f11d15846 service cinder] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:25.775687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8458408-11c2-44cb-8a9a-666c07f96d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:25.776558 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8458408-11c2-44cb-8a9a-666c07f96d2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:25.776964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8458408-11c2-44cb-8a9a-666c07f96d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:25.777249 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8458408-11c2-44cb-8a9a-666c07f96d2e admin admin] PATH_INFO: `/v3/users/b1869cd5b42b4846859cbd06ef868133` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:25.778488 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8458408-11c2-44cb-8a9a-666c07f96d2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1869cd5b42b4846859cbd06ef868133)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:25.780437 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8458408-11c2-44cb-8a9a-666c07f96d2e admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:25.956364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f8458408-11c2-44cb-8a9a-666c07f96d2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1869cd5b42b4846859cbd06ef868133'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:25.963595 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f8458408-11c2-44cb-8a9a-666c07f96d2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1869cd5b42b4846859cbd06ef868133'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:25.964272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f8458408-11c2-44cb-8a9a-666c07f96d2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1869cd5b42b4846859cbd06ef868133'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:25.976756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f8458408-11c2-44cb-8a9a-666c07f96d2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1869cd5b42b4846859cbd06ef868133'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:26.051252 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f8458408-11c2-44cb-8a9a-666c07f96d2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1869cd5b42b4846859cbd06ef868133'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:26.058837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1007/2030] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:46:25 2022] DELETE /identity/v3/users/b1869cd5b42b4846859cbd06ef868133 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:26.063694 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d447e5-888b-4134-85d6-a13c601769f5 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:26.064155 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d447e5-888b-4134-85d6-a13c601769f5 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:26.064267 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d447e5-888b-4134-85d6-a13c601769f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:26.082041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-92d447e5-888b-4134-85d6-a13c601769f5 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:26.083326 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-92d447e5-888b-4134-85d6-a13c601769f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:26.147767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1024/2031] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:26.157496 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8458408-11c2-44cb-8a9a-666c07f96d2e admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:26.159026 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f8458408-11c2-44cb-8a9a-666c07f96d2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:26.223372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8458408-11c2-44cb-8a9a-666c07f96d2e admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:26.228095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a96c29-319e-4452-a547-dc932514878a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:26.228749 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a96c29-319e-4452-a547-dc932514878a service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:26.229014 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a96c29-319e-4452-a547-dc932514878a service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:26.229289 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a96c29-319e-4452-a547-dc932514878a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:26.239729 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a96c29-319e-4452-a547-dc932514878a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:26.241864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a96c29-319e-4452-a547-dc932514878a service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:26.252509 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1008/2032] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:26.676892 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d447e5-888b-4134-85d6-a13c601769f5 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:26.678583 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-92d447e5-888b-4134-85d6-a13c601769f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:26.691112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d447e5-888b-4134-85d6-a13c601769f5 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:26.696009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0781d-725a-4cae-a55e-51b350e97a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:26.696928 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0781d-725a-4cae-a55e-51b350e97a4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:26.697211 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0781d-725a-4cae-a55e-51b350e97a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:26.697488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0781d-725a-4cae-a55e-51b350e97a4f admin admin] PATH_INFO: `/v3/projects/4a2dbe71f2c6477f8e4cc2d3d0248b19` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:26.698756 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f0781d-725a-4cae-a55e-51b350e97a4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a2dbe71f2c6477f8e4cc2d3d0248b19)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:26.700711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f0781d-725a-4cae-a55e-51b350e97a4f admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:26.807984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-a3f0781d-725a-4cae-a55e-51b350e97a4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a2dbe71f2c6477f8e4cc2d3d0248b19', 'request_id': 'req-a3f0781d-725a-4cae-a55e-51b350e97a4f'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:26.875430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-a3f0781d-725a-4cae-a55e-51b350e97a4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a2dbe71f2c6477f8e4cc2d3d0248b19', 'request_id': 'req-a3f0781d-725a-4cae-a55e-51b350e97a4f'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:26.882365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-a3f0781d-725a-4cae-a55e-51b350e97a4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a2dbe71f2c6477f8e4cc2d3d0248b19', 'request_id': 'req-a3f0781d-725a-4cae-a55e-51b350e97a4f'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:26.892722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-a3f0781d-725a-4cae-a55e-51b350e97a4f admin admin] The token cache is being invalidate because project 4a2dbe71f2c6477f8e4cc2d3d0248b19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:46:26.893009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-a3f0781d-725a-4cae-a55e-51b350e97a4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:26.893654 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-a3f0781d-725a-4cae-a55e-51b350e97a4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:26.895307 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1025/2033] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:46:26 2022] DELETE /identity/v3/projects/4a2dbe71f2c6477f8e4cc2d3d0248b19 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:26.900296 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a96c29-319e-4452-a547-dc932514878a service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:26.911094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a96c29-319e-4452-a547-dc932514878a service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:26.968458 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b532e-a803-4f40-a842-98b5a87b51bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:26.969330 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b532e-a803-4f40-a842-98b5a87b51bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:26.969712 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b532e-a803-4f40-a842-98b5a87b51bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:26.970011 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b532e-a803-4f40-a842-98b5a87b51bb admin admin] PATH_INFO: `/v3/users/bb4e46f4b08a489d9aec77bfb2b865b7` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:26.971241 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b532e-a803-4f40-a842-98b5a87b51bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb4e46f4b08a489d9aec77bfb2b865b7)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:26.973212 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b532e-a803-4f40-a842-98b5a87b51bb admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:26.995335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:27.153281 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-871b532e-a803-4f40-a842-98b5a87b51bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb4e46f4b08a489d9aec77bfb2b865b7'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:27.160013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-871b532e-a803-4f40-a842-98b5a87b51bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb4e46f4b08a489d9aec77bfb2b865b7'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:27.160700 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-871b532e-a803-4f40-a842-98b5a87b51bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb4e46f4b08a489d9aec77bfb2b865b7'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:27.172978 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-871b532e-a803-4f40-a842-98b5a87b51bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb4e46f4b08a489d9aec77bfb2b865b7'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:27.235618 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-871b532e-a803-4f40-a842-98b5a87b51bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb4e46f4b08a489d9aec77bfb2b865b7'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:27.242567 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1009/2034] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:46:26 2022] DELETE /identity/v3/users/bb4e46f4b08a489d9aec77bfb2b865b7 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:27.678019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0781d-725a-4cae-a55e-51b350e97a4f admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:27.679686 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-a3f0781d-725a-4cae-a55e-51b350e97a4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:27.693274 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0781d-725a-4cae-a55e-51b350e97a4f admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:27.755907 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca754c9-947c-498e-a0da-6a861f8f90d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:27.756765 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca754c9-947c-498e-a0da-6a861f8f90d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:27.757046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca754c9-947c-498e-a0da-6a861f8f90d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:27.757323 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca754c9-947c-498e-a0da-6a861f8f90d4 admin admin] PATH_INFO: `/v3/projects/f12950d215234ca5889adb775a5eb6bd` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:27.758538 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca754c9-947c-498e-a0da-6a861f8f90d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f12950d215234ca5889adb775a5eb6bd)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:27.760446 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca754c9-947c-498e-a0da-6a861f8f90d4 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:27.766663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:27.956742 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-eca754c9-947c-498e-a0da-6a861f8f90d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f12950d215234ca5889adb775a5eb6bd', 'request_id': 'req-eca754c9-947c-498e-a0da-6a861f8f90d4'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:28.064182 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-eca754c9-947c-498e-a0da-6a861f8f90d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f12950d215234ca5889adb775a5eb6bd', 'request_id': 'req-eca754c9-947c-498e-a0da-6a861f8f90d4'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:28.070944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-eca754c9-947c-498e-a0da-6a861f8f90d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f12950d215234ca5889adb775a5eb6bd', 'request_id': 'req-eca754c9-947c-498e-a0da-6a861f8f90d4'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:28.081829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-eca754c9-947c-498e-a0da-6a861f8f90d4 admin admin] The token cache is being invalidate because project f12950d215234ca5889adb775a5eb6bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:46:28.082117 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-eca754c9-947c-498e-a0da-6a861f8f90d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:28.082813 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-eca754c9-947c-498e-a0da-6a861f8f90d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:28.085019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1026/2035] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:46:27 2022] DELETE /identity/v3/projects/f12950d215234ca5889adb775a5eb6bd => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:28.089896 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bff40b-8c4d-46a5-873e-5aba6e81bfe8 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:28.090372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bff40b-8c4d-46a5-873e-5aba6e81bfe8 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:28.090521 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bff40b-8c4d-46a5-873e-5aba6e81bfe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:28.109544 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-17bff40b-8c4d-46a5-873e-5aba6e81bfe8 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:28.110785 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-17bff40b-8c4d-46a5-873e-5aba6e81bfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:28.169178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1010/2036] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:28.176723 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ceb755-9ffd-4ffa-9b4b-14e7562dc92c None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:28.177008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ceb755-9ffd-4ffa-9b4b-14e7562dc92c None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:28.177308 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ceb755-9ffd-4ffa-9b4b-14e7562dc92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:28.195314 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-a7ceb755-9ffd-4ffa-9b4b-14e7562dc92c None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:28.197111 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-a7ceb755-9ffd-4ffa-9b4b-14e7562dc92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:28.204798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1027/2037] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:28.209716 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bff40b-8c4d-46a5-873e-5aba6e81bfe8 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:28.219652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bff40b-8c4d-46a5-873e-5aba6e81bfe8 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:28.224411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01ac1c-870b-4bc5-a73c-b8d9e2825f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:28.225858 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01ac1c-870b-4bc5-a73c-b8d9e2825f12 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:28.226144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01ac1c-870b-4bc5-a73c-b8d9e2825f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:28.226456 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01ac1c-870b-4bc5-a73c-b8d9e2825f12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:28.227194 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb01ac1c-870b-4bc5-a73c-b8d9e2825f12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:46:28.227548 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb01ac1c-870b-4bc5-a73c-b8d9e2825f12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:28.229145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb01ac1c-870b-4bc5-a73c-b8d9e2825f12 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:28.241441 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1011/2038] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:46:28.245629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ceb755-9ffd-4ffa-9b4b-14e7562dc92c None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:28.255231 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ceb755-9ffd-4ffa-9b4b-14e7562dc92c None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:28.260166 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183f54e-8157-43fe-b3d1-c04c24893287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:28.263418 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183f54e-8157-43fe-b3d1-c04c24893287 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:28.263418 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183f54e-8157-43fe-b3d1-c04c24893287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:28.263418 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183f54e-8157-43fe-b3d1-c04c24893287 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:28.263418 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183f54e-8157-43fe-b3d1-c04c24893287 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:28.263940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183f54e-8157-43fe-b3d1-c04c24893287 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:28.381291 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1028/2039] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:46:28 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:28.385477 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01ac1c-870b-4bc5-a73c-b8d9e2825f12 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:28.395867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01ac1c-870b-4bc5-a73c-b8d9e2825f12 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:28.454571 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc257fa-414f-4317-9725-878694572c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:28.455235 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc257fa-414f-4317-9725-878694572c4d admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:28.455514 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc257fa-414f-4317-9725-878694572c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:28.455785 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc257fa-414f-4317-9725-878694572c4d admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:28.456514 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc257fa-414f-4317-9725-878694572c4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:28.458446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc257fa-414f-4317-9725-878694572c4d admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:28.538736 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1012/2040] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:46:28 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:46:28.542900 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183f54e-8157-43fe-b3d1-c04c24893287 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:28.553995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183f54e-8157-43fe-b3d1-c04c24893287 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:28.558578 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbf0cc-ea6c-4996-8b3f-ce97fa839846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:28.559264 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbf0cc-ea6c-4996-8b3f-ce97fa839846 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:28.559544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbf0cc-ea6c-4996-8b3f-ce97fa839846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:28.559814 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbf0cc-ea6c-4996-8b3f-ce97fa839846 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:28.560534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efbf0cc-ea6c-4996-8b3f-ce97fa839846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:28.562119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efbf0cc-ea6c-4996-8b3f-ce97fa839846 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:28.577456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1029/2041] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:46:28.578744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc257fa-414f-4317-9725-878694572c4d admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:28.590157 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc257fa-414f-4317-9725-878694572c4d admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:28.595012 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf9a60-f8d5-434f-bbc7-c6e6dfe59620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:28.595670 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf9a60-f8d5-434f-bbc7-c6e6dfe59620 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:28.595955 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf9a60-f8d5-434f-bbc7-c6e6dfe59620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:28.596230 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf9a60-f8d5-434f-bbc7-c6e6dfe59620 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:28.597038 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf9a60-f8d5-434f-bbc7-c6e6dfe59620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:28.598694 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf9a60-f8d5-434f-bbc7-c6e6dfe59620 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:28.616473 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1013/2042] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:28.622543 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbf0cc-ea6c-4996-8b3f-ce97fa839846 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:28.631703 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbf0cc-ea6c-4996-8b3f-ce97fa839846 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:28.636400 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ed5c9-3a1f-47db-84c7-371c957ee54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:28.637282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ed5c9-3a1f-47db-84c7-371c957ee54c admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:28.637565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ed5c9-3a1f-47db-84c7-371c957ee54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:28.637842 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ed5c9-3a1f-47db-84c7-371c957ee54c admin admin] PATH_INFO: `/v3/projects/753b5d2157944073b351fe3eb6b0b622/users/0fc43db8323043a0a066f2be1eb2a9f6/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:28.654171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ed5c9-3a1f-47db-84c7-371c957ee54c admin admin] RBAC: Authorizing `identity:create_grant(project_id=753b5d2157944073b351fe3eb6b0b622, user_id=0fc43db8323043a0a066f2be1eb2a9f6, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:28.656980 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ed5c9-3a1f-47db-84c7-371c957ee54c admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:28.758603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1030/2043] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:28 2022] PUT /identity/v3/projects/753b5d2157944073b351fe3eb6b0b622/users/0fc43db8323043a0a066f2be1eb2a9f6/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:28.770100 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88028dd3-2486-4645-8d43-a414900f1415 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:28.770416 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88028dd3-2486-4645-8d43-a414900f1415 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:28.770696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88028dd3-2486-4645-8d43-a414900f1415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:28.780853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:28.805898 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-88028dd3-2486-4645-8d43-a414900f1415 None None] MFA Rules not processed for user `0fc43db8323043a0a066f2be1eb2a9f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:28.807837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-88028dd3-2486-4645-8d43-a414900f1415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:28.872684 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1014/2044] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:46:28.877377 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ed5c9-3a1f-47db-84c7-371c957ee54c admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:28.940404 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ed5c9-3a1f-47db-84c7-371c957ee54c admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:28.946079 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584e801-f400-45a0-b219-510eb948e4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:28.946753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584e801-f400-45a0-b219-510eb948e4f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:28.947029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584e801-f400-45a0-b219-510eb948e4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:28.947302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584e801-f400-45a0-b219-510eb948e4f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:28.947973 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7584e801-f400-45a0-b219-510eb948e4f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:28.949862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7584e801-f400-45a0-b219-510eb948e4f6 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:29.028695 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1031/2045] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:46:28 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:29.033050 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88028dd3-2486-4645-8d43-a414900f1415 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:29.044046 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88028dd3-2486-4645-8d43-a414900f1415 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:29.104969 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98920d13-569c-47d5-a005-9d4c80ddbb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:29.105675 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98920d13-569c-47d5-a005-9d4c80ddbb07 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:29.106612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98920d13-569c-47d5-a005-9d4c80ddbb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:29.106612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98920d13-569c-47d5-a005-9d4c80ddbb07 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:29.107264 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98920d13-569c-47d5-a005-9d4c80ddbb07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:29.109273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98920d13-569c-47d5-a005-9d4c80ddbb07 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:29.207702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1015/2046] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:46:29 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:29.212148 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584e801-f400-45a0-b219-510eb948e4f6 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:29.222403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584e801-f400-45a0-b219-510eb948e4f6 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:29.227806 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1996e-c2c0-493b-8cde-95ca13c4423b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:29.228459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1996e-c2c0-493b-8cde-95ca13c4423b admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:29.231619 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1996e-c2c0-493b-8cde-95ca13c4423b admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:29.231619 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1996e-c2c0-493b-8cde-95ca13c4423b admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:29.231619 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1996e-c2c0-493b-8cde-95ca13c4423b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:29.231894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1996e-c2c0-493b-8cde-95ca13c4423b admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:29.247957 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1032/2047] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:29.249121 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98920d13-569c-47d5-a005-9d4c80ddbb07 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:29.260236 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98920d13-569c-47d5-a005-9d4c80ddbb07 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:29.265185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6521d-54f0-454b-838a-67bf5726c011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:29.265833 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6521d-54f0-454b-838a-67bf5726c011 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:29.266121 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6521d-54f0-454b-838a-67bf5726c011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:29.266404 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6521d-54f0-454b-838a-67bf5726c011 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:29.267144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da6521d-54f0-454b-838a-67bf5726c011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:29.268772 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da6521d-54f0-454b-838a-67bf5726c011 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:29.279265 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1016/2048] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:46:29.283426 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1996e-c2c0-493b-8cde-95ca13c4423b admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:29.290410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:29.294294 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1996e-c2c0-493b-8cde-95ca13c4423b admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:29.299010 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc4dae-8f07-4585-be80-4297a2f7a152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:29.299820 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc4dae-8f07-4585-be80-4297a2f7a152 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:29.300108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc4dae-8f07-4585-be80-4297a2f7a152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:29.300387 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc4dae-8f07-4585-be80-4297a2f7a152 admin admin] PATH_INFO: `/v3/projects/ebc05582a9d547c5aca8136030e5f322/users/1286d65b189745e5b49d83f6eebbd8bd/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:29.315847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc4dae-8f07-4585-be80-4297a2f7a152 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebc05582a9d547c5aca8136030e5f322, user_id=1286d65b189745e5b49d83f6eebbd8bd, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:29.318695 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc4dae-8f07-4585-be80-4297a2f7a152 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:29.425479 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1033/2049] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:29 2022] PUT /identity/v3/projects/ebc05582a9d547c5aca8136030e5f322/users/1286d65b189745e5b49d83f6eebbd8bd/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:29.435666 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a430b-25c9-4fa4-b51a-1c004035d393 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:29.435961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a430b-25c9-4fa4-b51a-1c004035d393 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:29.436301 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a430b-25c9-4fa4-b51a-1c004035d393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:29.467601 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-9a0a430b-25c9-4fa4-b51a-1c004035d393 None None] MFA Rules not processed for user `1286d65b189745e5b49d83f6eebbd8bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:29.468885 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-9a0a430b-25c9-4fa4-b51a-1c004035d393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:29.536385 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1017/2050] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:29.541202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc4dae-8f07-4585-be80-4297a2f7a152 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:29.603868 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc4dae-8f07-4585-be80-4297a2f7a152 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:29.608799 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7ce57-2d8f-414a-ae36-27a823116d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:29.609450 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7ce57-2d8f-414a-ae36-27a823116d59 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:29.609734 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7ce57-2d8f-414a-ae36-27a823116d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:29.610039 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7ce57-2d8f-414a-ae36-27a823116d59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:29.610780 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7ce57-2d8f-414a-ae36-27a823116d59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:29.612680 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7ce57-2d8f-414a-ae36-27a823116d59 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:29.720552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1034/2051] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:46:29 2022] POST /identity/v3/projects => generated 383 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:46:29.721969 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a430b-25c9-4fa4-b51a-1c004035d393 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:29.727805 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:29.748731 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a430b-25c9-4fa4-b51a-1c004035d393 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:29.839393 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fbd2a-6027-4798-91d5-f2bcdbd135dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:29.851007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fbd2a-6027-4798-91d5-f2bcdbd135dc admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:29.851007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fbd2a-6027-4798-91d5-f2bcdbd135dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:29.851007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fbd2a-6027-4798-91d5-f2bcdbd135dc admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:29.851007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498fbd2a-6027-4798-91d5-f2bcdbd135dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:29.851007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498fbd2a-6027-4798-91d5-f2bcdbd135dc admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:29.930492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1018/2052] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:46:29 2022] POST /identity/v3/users => generated 298 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:46:29.934886 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7ce57-2d8f-414a-ae36-27a823116d59 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:29.960293 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7ce57-2d8f-414a-ae36-27a823116d59 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:29.967374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876d917-87de-494c-862d-0d534b8710ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:29.968199 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876d917-87de-494c-862d-0d534b8710ed admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:29.968641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876d917-87de-494c-862d-0d534b8710ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:29.969013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876d917-87de-494c-862d-0d534b8710ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:29.970028 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9876d917-87de-494c-862d-0d534b8710ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:29.972232 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9876d917-87de-494c-862d-0d534b8710ed admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:29.986672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1035/2053] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:46:29.992191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fbd2a-6027-4798-91d5-f2bcdbd135dc admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:30.004933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fbd2a-6027-4798-91d5-f2bcdbd135dc admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:30.010982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb15baa-a3be-4722-95a9-efe8d11701b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:30.012316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb15baa-a3be-4722-95a9-efe8d11701b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:30.012316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb15baa-a3be-4722-95a9-efe8d11701b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:30.012700 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb15baa-a3be-4722-95a9-efe8d11701b5 admin admin] PATH_INFO: `/v3/projects/d55a30e38f534f679e221b42ffec616c/users/0b005e3d155b4657b0a7d05d43081afc/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:30.031660 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb15baa-a3be-4722-95a9-efe8d11701b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d55a30e38f534f679e221b42ffec616c, user_id=0b005e3d155b4657b0a7d05d43081afc, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:30.034366 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb15baa-a3be-4722-95a9-efe8d11701b5 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:30.128206 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1019/2054] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:29 2022] PUT /identity/v3/projects/d55a30e38f534f679e221b42ffec616c/users/0b005e3d155b4657b0a7d05d43081afc/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:30.133246 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876d917-87de-494c-862d-0d534b8710ed admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:30.143456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876d917-87de-494c-862d-0d534b8710ed admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:30.204097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a8845-7fbb-4d2a-8776-e6d9951d66f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:30.204097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a8845-7fbb-4d2a-8776-e6d9951d66f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:30.204361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a8845-7fbb-4d2a-8776-e6d9951d66f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:30.204469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a8845-7fbb-4d2a-8776-e6d9951d66f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:30.205136 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a8845-7fbb-4d2a-8776-e6d9951d66f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:30.206803 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a8845-7fbb-4d2a-8776-e6d9951d66f0 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:30.218183 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1036/2055] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:30.228193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb15baa-a3be-4722-95a9-efe8d11701b5 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:30.238700 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb15baa-a3be-4722-95a9-efe8d11701b5 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:30.244624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf6a22-2e4f-4f95-aa53-55337ede9554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:30.245507 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf6a22-2e4f-4f95-aa53-55337ede9554 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:30.245815 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf6a22-2e4f-4f95-aa53-55337ede9554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:30.246108 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf6a22-2e4f-4f95-aa53-55337ede9554 admin admin] PATH_INFO: `/v3/projects/d55a30e38f534f679e221b42ffec616c/users/0b005e3d155b4657b0a7d05d43081afc/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:30.248149 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf6a22-2e4f-4f95-aa53-55337ede9554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d55a30e38f534f679e221b42ffec616c, user_id=0b005e3d155b4657b0a7d05d43081afc, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:30.251638 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf6a22-2e4f-4f95-aa53-55337ede9554 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:30.261796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1020/2056] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:30 2022] PUT /identity/v3/projects/d55a30e38f534f679e221b42ffec616c/users/0b005e3d155b4657b0a7d05d43081afc/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:30.272024 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02e1d2-d87d-42cb-90d7-73ef10740b58 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:30.274519 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02e1d2-d87d-42cb-90d7-73ef10740b58 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:30.274519 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02e1d2-d87d-42cb-90d7-73ef10740b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:30.298722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-bc02e1d2-d87d-42cb-90d7-73ef10740b58 None None] MFA Rules not processed for user `0b005e3d155b4657b0a7d05d43081afc`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:30.299927 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-bc02e1d2-d87d-42cb-90d7-73ef10740b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:30.340944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:30.368955 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1037/2057] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:30.380119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf6a22-2e4f-4f95-aa53-55337ede9554 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:30.382708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-fedf6a22-2e4f-4f95-aa53-55337ede9554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:30.440303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf6a22-2e4f-4f95-aa53-55337ede9554 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:30.444698 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895c6a5-06ef-4844-aa85-a04aee06c793 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:30.445346 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895c6a5-06ef-4844-aa85-a04aee06c793 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:30.445628 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895c6a5-06ef-4844-aa85-a04aee06c793 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:30.445909 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895c6a5-06ef-4844-aa85-a04aee06c793 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:30.504620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895c6a5-06ef-4844-aa85-a04aee06c793 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:30.507149 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895c6a5-06ef-4844-aa85-a04aee06c793 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:30.516537 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1021/2058] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:30.984039 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02e1d2-d87d-42cb-90d7-73ef10740b58 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:30.986129 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-bc02e1d2-d87d-42cb-90d7-73ef10740b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:31.088720 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02e1d2-d87d-42cb-90d7-73ef10740b58 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:31.094363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229c40a-835e-4be9-b89d-5eb08f9c444c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:31.095395 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229c40a-835e-4be9-b89d-5eb08f9c444c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:31.098833 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229c40a-835e-4be9-b89d-5eb08f9c444c admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:31.098833 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229c40a-835e-4be9-b89d-5eb08f9c444c admin admin] PATH_INFO: `/v3/users/eb023516388a4664a4dedb936020c8ed` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:31.101554 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7229c40a-835e-4be9-b89d-5eb08f9c444c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb023516388a4664a4dedb936020c8ed)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:31.104432 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7229c40a-835e-4be9-b89d-5eb08f9c444c admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:31.370431 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7229c40a-835e-4be9-b89d-5eb08f9c444c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb023516388a4664a4dedb936020c8ed'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:31.379189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7229c40a-835e-4be9-b89d-5eb08f9c444c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb023516388a4664a4dedb936020c8ed'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:31.379875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7229c40a-835e-4be9-b89d-5eb08f9c444c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb023516388a4664a4dedb936020c8ed'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:31.381105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7229c40a-835e-4be9-b89d-5eb08f9c444c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb023516388a4664a4dedb936020c8ed'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:31.395755 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7229c40a-835e-4be9-b89d-5eb08f9c444c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb023516388a4664a4dedb936020c8ed'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:31.528453 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1038/2059] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:46:30 2022] DELETE /identity/v3/users/eb023516388a4664a4dedb936020c8ed => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:31.534864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932e974-3540-48a5-b1fe-87bb6220e5ac None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:31.534864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932e974-3540-48a5-b1fe-87bb6220e5ac None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:31.535406 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932e974-3540-48a5-b1fe-87bb6220e5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:31.560669 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-9932e974-3540-48a5-b1fe-87bb6220e5ac None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:31.562302 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-9932e974-3540-48a5-b1fe-87bb6220e5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:31.568542 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:31.639096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1022/2060] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:46:31.657465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229c40a-835e-4be9-b89d-5eb08f9c444c admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:31.657465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-7229c40a-835e-4be9-b89d-5eb08f9c444c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:31.716764 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229c40a-835e-4be9-b89d-5eb08f9c444c admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:31.720630 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c12f4-495b-4623-b2a8-b051bba8beaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:31.721340 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c12f4-495b-4623-b2a8-b051bba8beaa service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:31.721791 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c12f4-495b-4623-b2a8-b051bba8beaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:31.722137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c12f4-495b-4623-b2a8-b051bba8beaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:31.732643 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c12f4-495b-4623-b2a8-b051bba8beaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:31.734077 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c12f4-495b-4623-b2a8-b051bba8beaa service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:31.738422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:31.754603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1039/2061] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:32.230506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932e974-3540-48a5-b1fe-87bb6220e5ac None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:32.232102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-9932e974-3540-48a5-b1fe-87bb6220e5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:32.247146 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932e974-3540-48a5-b1fe-87bb6220e5ac None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:32.251859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12666a-bb65-4fdf-8853-c4557d253945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:32.252765 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12666a-bb65-4fdf-8853-c4557d253945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:32.253043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12666a-bb65-4fdf-8853-c4557d253945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:32.255141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12666a-bb65-4fdf-8853-c4557d253945 admin admin] PATH_INFO: `/v3/projects/0343ff062aba4b67b28b2ecf2493d2f1` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:32.255141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12666a-bb65-4fdf-8853-c4557d253945 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0343ff062aba4b67b28b2ecf2493d2f1)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:32.257401 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12666a-bb65-4fdf-8853-c4557d253945 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:32.384934 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-ab12666a-bb65-4fdf-8853-c4557d253945 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0343ff062aba4b67b28b2ecf2493d2f1', 'request_id': 'req-ab12666a-bb65-4fdf-8853-c4557d253945'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:32.459300 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-ab12666a-bb65-4fdf-8853-c4557d253945 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0343ff062aba4b67b28b2ecf2493d2f1', 'request_id': 'req-ab12666a-bb65-4fdf-8853-c4557d253945'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:32.465984 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-ab12666a-bb65-4fdf-8853-c4557d253945 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0343ff062aba4b67b28b2ecf2493d2f1', 'request_id': 'req-ab12666a-bb65-4fdf-8853-c4557d253945'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:32.477188 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-ab12666a-bb65-4fdf-8853-c4557d253945 admin admin] The token cache is being invalidate because project 0343ff062aba4b67b28b2ecf2493d2f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:46:32.480883 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-ab12666a-bb65-4fdf-8853-c4557d253945 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:32.480883 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-ab12666a-bb65-4fdf-8853-c4557d253945 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:32.480883 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1023/2062] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:46:32 2022] DELETE /identity/v3/projects/0343ff062aba4b67b28b2ecf2493d2f1 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:32.485055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa9af8-cfa5-4706-ba62-ea8d10d991fc None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:32.485570 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa9af8-cfa5-4706-ba62-ea8d10d991fc None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:32.485784 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa9af8-cfa5-4706-ba62-ea8d10d991fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:32.506216 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-e7aa9af8-cfa5-4706-ba62-ea8d10d991fc None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:32.508043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-e7aa9af8-cfa5-4706-ba62-ea8d10d991fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:32.570100 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1040/2063] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:32.577960 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8afa95-6cb7-4411-8025-43389f078424 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:32.578475 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8afa95-6cb7-4411-8025-43389f078424 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:32.578690 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8afa95-6cb7-4411-8025-43389f078424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:32.598840 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-1a8afa95-6cb7-4411-8025-43389f078424 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:32.600811 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-1a8afa95-6cb7-4411-8025-43389f078424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:32.609693 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1024/2064] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:32.613974 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa9af8-cfa5-4706-ba62-ea8d10d991fc None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:32.623916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa9af8-cfa5-4706-ba62-ea8d10d991fc None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:32.628666 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315e83e-4ca0-4408-996a-f7b3ffc85d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:32.629847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315e83e-4ca0-4408-996a-f7b3ffc85d37 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:32.629847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315e83e-4ca0-4408-996a-f7b3ffc85d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:32.630030 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315e83e-4ca0-4408-996a-f7b3ffc85d37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:32.630764 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b315e83e-4ca0-4408-996a-f7b3ffc85d37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:46:32.631208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b315e83e-4ca0-4408-996a-f7b3ffc85d37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:32.632655 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b315e83e-4ca0-4408-996a-f7b3ffc85d37 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:32.645749 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1041/2065] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:46:32.650787 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8afa95-6cb7-4411-8025-43389f078424 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:32.661292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8afa95-6cb7-4411-8025-43389f078424 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:32.666806 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e294ec5-22b2-462a-b1c8-b7112af076da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:32.667463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e294ec5-22b2-462a-b1c8-b7112af076da admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:32.667745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e294ec5-22b2-462a-b1c8-b7112af076da admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:32.668222 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e294ec5-22b2-462a-b1c8-b7112af076da admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:32.668916 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e294ec5-22b2-462a-b1c8-b7112af076da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:32.672291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e294ec5-22b2-462a-b1c8-b7112af076da admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:32.768855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1025/2066] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:46:32 2022] POST /identity/v3/projects => generated 399 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:32.773523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315e83e-4ca0-4408-996a-f7b3ffc85d37 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:32.783597 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315e83e-4ca0-4408-996a-f7b3ffc85d37 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:32.841623 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e0601-a677-4b3d-920c-3ce92f6a028f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:32.842276 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e0601-a677-4b3d-920c-3ce92f6a028f admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:32.842583 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e0601-a677-4b3d-920c-3ce92f6a028f admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:32.842860 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e0601-a677-4b3d-920c-3ce92f6a028f admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:32.843534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e0601-a677-4b3d-920c-3ce92f6a028f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:32.845423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e0601-a677-4b3d-920c-3ce92f6a028f admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:32.965985 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1042/2067] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:46:32 2022] POST /identity/v3/users => generated 300 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:32.970185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e294ec5-22b2-462a-b1c8-b7112af076da admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:32.983582 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e294ec5-22b2-462a-b1c8-b7112af076da admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:32.988248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d6c85-60cf-4a5f-9a7c-8a11cc77e6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:32.988906 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d6c85-60cf-4a5f-9a7c-8a11cc77e6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:32.989198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d6c85-60cf-4a5f-9a7c-8a11cc77e6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:32.989477 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d6c85-60cf-4a5f-9a7c-8a11cc77e6d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:32.990286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d6c85-60cf-4a5f-9a7c-8a11cc77e6d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:32.991925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d6c85-60cf-4a5f-9a7c-8a11cc77e6d5 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:33.002422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1026/2068] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:33.007111 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e0601-a677-4b3d-920c-3ce92f6a028f admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:33.017237 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e0601-a677-4b3d-920c-3ce92f6a028f admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:33.023409 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8efc459-dc36-4600-a997-9d7e52dd1410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:33.024331 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efc459-dc36-4600-a997-9d7e52dd1410 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:33.024792 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efc459-dc36-4600-a997-9d7e52dd1410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:33.025236 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efc459-dc36-4600-a997-9d7e52dd1410 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:33.026459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8efc459-dc36-4600-a997-9d7e52dd1410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:33.029375 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8efc459-dc36-4600-a997-9d7e52dd1410 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:33.044040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1043/2069] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:33.048383 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d6c85-60cf-4a5f-9a7c-8a11cc77e6d5 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:33.058482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d6c85-60cf-4a5f-9a7c-8a11cc77e6d5 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:33.063122 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4abf02-c98c-4596-b0c3-af5ff79f4257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:33.063987 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4abf02-c98c-4596-b0c3-af5ff79f4257 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:33.064272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4abf02-c98c-4596-b0c3-af5ff79f4257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:33.064549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4abf02-c98c-4596-b0c3-af5ff79f4257 admin admin] PATH_INFO: `/v3/projects/ea3a07549a8c41eeb4604f65d39ae33e/users/b2b08a473fe64a8d964507711e21f8d7/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:33.080489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4abf02-c98c-4596-b0c3-af5ff79f4257 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea3a07549a8c41eeb4604f65d39ae33e, user_id=b2b08a473fe64a8d964507711e21f8d7, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:33.083262 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4abf02-c98c-4596-b0c3-af5ff79f4257 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:33.154444 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1027/2070] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:33 2022] PUT /identity/v3/projects/ea3a07549a8c41eeb4604f65d39ae33e/users/b2b08a473fe64a8d964507711e21f8d7/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:46:33.165458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3abe17-0fac-4f79-8798-fd50260d78c8 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:33.165765 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3abe17-0fac-4f79-8798-fd50260d78c8 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:33.166061 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3abe17-0fac-4f79-8798-fd50260d78c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:33.195606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-ac3abe17-0fac-4f79-8798-fd50260d78c8 None None] MFA Rules not processed for user `b2b08a473fe64a8d964507711e21f8d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:33.197889 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ac3abe17-0fac-4f79-8798-fd50260d78c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:33.269771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:33.283345 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1044/2071] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:46:33.296456 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4abf02-c98c-4596-b0c3-af5ff79f4257 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:33.301220 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-3a4abf02-c98c-4596-b0c3-af5ff79f4257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:33.353821 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:33.382211 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4abf02-c98c-4596-b0c3-af5ff79f4257 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:33.385942 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca277e-2877-405d-a647-4b7af8b3c2b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:33.386670 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca277e-2877-405d-a647-4b7af8b3c2b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:33.386936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca277e-2877-405d-a647-4b7af8b3c2b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:33.387218 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca277e-2877-405d-a647-4b7af8b3c2b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:33.396622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca277e-2877-405d-a647-4b7af8b3c2b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:33.398200 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca277e-2877-405d-a647-4b7af8b3c2b0 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:33.408366 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1028/2072] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:33 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:34.486009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3abe17-0fac-4f79-8798-fd50260d78c8 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:34.501572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3abe17-0fac-4f79-8798-fd50260d78c8 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:34.508683 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815e0df-7f2a-4643-bfd4-6743982ff1ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:34.509412 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815e0df-7f2a-4643-bfd4-6743982ff1ee service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:34.509761 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815e0df-7f2a-4643-bfd4-6743982ff1ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:34.510176 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815e0df-7f2a-4643-bfd4-6743982ff1ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:34.512145 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-e815e0df-7f2a-4643-bfd4-6743982ff1ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:34.585404 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e815e0df-7f2a-4643-bfd4-6743982ff1ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:34.587030 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e815e0df-7f2a-4643-bfd4-6743982ff1ee service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:34.597106 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1045/2073] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:34.756949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca277e-2877-405d-a647-4b7af8b3c2b0 service cinder] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:34.758472 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-08ca277e-2877-405d-a647-4b7af8b3c2b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:34.770399 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca277e-2877-405d-a647-4b7af8b3c2b0 service cinder] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:34.774728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729440b-e967-42e0-a026-0f1cd678c748 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:34.775366 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729440b-e967-42e0-a026-0f1cd678c748 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:34.775679 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729440b-e967-42e0-a026-0f1cd678c748 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:34.775958 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729440b-e967-42e0-a026-0f1cd678c748 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:34.777880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-8729440b-e967-42e0-a026-0f1cd678c748 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:34.846403 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729440b-e967-42e0-a026-0f1cd678c748 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:34.847854 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729440b-e967-42e0-a026-0f1cd678c748 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:34.858948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1029/2074] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:46:37.306138 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815e0df-7f2a-4643-bfd4-6743982ff1ee service cinder] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:37.319868 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815e0df-7f2a-4643-bfd4-6743982ff1ee service cinder] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:37.319868 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175ad2a7-bf3f-48e0-aa6e-c6506bf952ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:37.320051 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ad2a7-bf3f-48e0-aa6e-c6506bf952ba service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:37.320227 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ad2a7-bf3f-48e0-aa6e-c6506bf952ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:37.320505 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175ad2a7-bf3f-48e0-aa6e-c6506bf952ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:37.322461 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-175ad2a7-bf3f-48e0-aa6e-c6506bf952ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:37.390130 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ad2a7-bf3f-48e0-aa6e-c6506bf952ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:37.391693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175ad2a7-bf3f-48e0-aa6e-c6506bf952ba service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:37.401593 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1046/2075] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:37 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:37.683922 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729440b-e967-42e0-a026-0f1cd678c748 service cinder] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:37.685353 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-8729440b-e967-42e0-a026-0f1cd678c748 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:37.751515 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729440b-e967-42e0-a026-0f1cd678c748 service cinder] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:37.757096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94122a5-a4e7-4bf3-a08f-27674838401f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:37.757897 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94122a5-a4e7-4bf3-a08f-27674838401f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:37.758175 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94122a5-a4e7-4bf3-a08f-27674838401f admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:37.758482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94122a5-a4e7-4bf3-a08f-27674838401f admin admin] PATH_INFO: `/v3/users/deeed099bc2b40d1b43f6ef1a03d0541` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:37.759668 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94122a5-a4e7-4bf3-a08f-27674838401f admin admin] RBAC: Authorizing `identity:delete_user(user_id=deeed099bc2b40d1b43f6ef1a03d0541)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:37.761579 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94122a5-a4e7-4bf3-a08f-27674838401f admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:37.932035 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-d94122a5-a4e7-4bf3-a08f-27674838401f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deeed099bc2b40d1b43f6ef1a03d0541'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:37.938986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-d94122a5-a4e7-4bf3-a08f-27674838401f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deeed099bc2b40d1b43f6ef1a03d0541'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:37.939668 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-d94122a5-a4e7-4bf3-a08f-27674838401f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deeed099bc2b40d1b43f6ef1a03d0541'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:37.952362 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:37.953772 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-d94122a5-a4e7-4bf3-a08f-27674838401f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deeed099bc2b40d1b43f6ef1a03d0541'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:38.067552 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-d94122a5-a4e7-4bf3-a08f-27674838401f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deeed099bc2b40d1b43f6ef1a03d0541'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:38.074655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1030/2076] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:46:37 2022] DELETE /identity/v3/users/deeed099bc2b40d1b43f6ef1a03d0541 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:38.079318 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e39d5-02c2-4deb-b18e-7518ef0d5345 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:38.079681 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e39d5-02c2-4deb-b18e-7518ef0d5345 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:38.080034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e39d5-02c2-4deb-b18e-7518ef0d5345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:38.097113 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-875e39d5-02c2-4deb-b18e-7518ef0d5345 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:38.098307 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-875e39d5-02c2-4deb-b18e-7518ef0d5345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:38.156920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1047/2077] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:38.168288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94122a5-a4e7-4bf3-a08f-27674838401f admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:38.170487 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-d94122a5-a4e7-4bf3-a08f-27674838401f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:38.231938 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94122a5-a4e7-4bf3-a08f-27674838401f admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:38.235577 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb9ddc-e519-4a28-bf3e-5affbd797881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:38.236214 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb9ddc-e519-4a28-bf3e-5affbd797881 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:38.236494 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb9ddc-e519-4a28-bf3e-5affbd797881 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:38.236769 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb9ddc-e519-4a28-bf3e-5affbd797881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:38.246232 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb9ddc-e519-4a28-bf3e-5affbd797881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:38.247691 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb9ddc-e519-4a28-bf3e-5affbd797881 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:38.257346 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1031/2078] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:46:38.681078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e39d5-02c2-4deb-b18e-7518ef0d5345 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:38.683047 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-875e39d5-02c2-4deb-b18e-7518ef0d5345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:38.697910 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875e39d5-02c2-4deb-b18e-7518ef0d5345 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:38.706960 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d34a7b-857b-415c-a260-3a197e448866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:38.706960 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d34a7b-857b-415c-a260-3a197e448866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:38.706960 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d34a7b-857b-415c-a260-3a197e448866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:38.706960 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d34a7b-857b-415c-a260-3a197e448866 admin admin] PATH_INFO: `/v3/projects/09af9f0756c84b5787f9a184b3aff8a3` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:38.707303 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d34a7b-857b-415c-a260-3a197e448866 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09af9f0756c84b5787f9a184b3aff8a3)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:38.708243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d34a7b-857b-415c-a260-3a197e448866 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:38.733827 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:38.833030 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-a0d34a7b-857b-415c-a260-3a197e448866 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09af9f0756c84b5787f9a184b3aff8a3', 'request_id': 'req-a0d34a7b-857b-415c-a260-3a197e448866'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:38.889790 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-a0d34a7b-857b-415c-a260-3a197e448866 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09af9f0756c84b5787f9a184b3aff8a3', 'request_id': 'req-a0d34a7b-857b-415c-a260-3a197e448866'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:38.896255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-a0d34a7b-857b-415c-a260-3a197e448866 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09af9f0756c84b5787f9a184b3aff8a3', 'request_id': 'req-a0d34a7b-857b-415c-a260-3a197e448866'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:38.907284 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-a0d34a7b-857b-415c-a260-3a197e448866 admin admin] The token cache is being invalidate because project 09af9f0756c84b5787f9a184b3aff8a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:46:38.907511 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-a0d34a7b-857b-415c-a260-3a197e448866 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:38.908159 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-a0d34a7b-857b-415c-a260-3a197e448866 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:38.909701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1048/2079] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:46:38 2022] DELETE /identity/v3/projects/09af9f0756c84b5787f9a184b3aff8a3 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:38.913604 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb9ddc-e519-4a28-bf3e-5affbd797881 service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:38.923214 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb9ddc-e519-4a28-bf3e-5affbd797881 service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:38.979763 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130e4f3-8eb2-4d60-ba2e-50adaccf090c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:38.980626 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130e4f3-8eb2-4d60-ba2e-50adaccf090c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:38.980911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130e4f3-8eb2-4d60-ba2e-50adaccf090c admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:38.981469 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130e4f3-8eb2-4d60-ba2e-50adaccf090c admin admin] PATH_INFO: `/v3/users/ca102d4720334b548b089adb37f0cf91` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:38.982699 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130e4f3-8eb2-4d60-ba2e-50adaccf090c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca102d4720334b548b089adb37f0cf91)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:38.984679 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130e4f3-8eb2-4d60-ba2e-50adaccf090c admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:39.137813 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-7130e4f3-8eb2-4d60-ba2e-50adaccf090c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca102d4720334b548b089adb37f0cf91'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:39.145606 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-7130e4f3-8eb2-4d60-ba2e-50adaccf090c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca102d4720334b548b089adb37f0cf91'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:39.147159 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-7130e4f3-8eb2-4d60-ba2e-50adaccf090c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca102d4720334b548b089adb37f0cf91'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:39.161085 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-7130e4f3-8eb2-4d60-ba2e-50adaccf090c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca102d4720334b548b089adb37f0cf91'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:39.247057 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-7130e4f3-8eb2-4d60-ba2e-50adaccf090c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca102d4720334b548b089adb37f0cf91'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:39.256770 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1032/2080] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:46:38 2022] DELETE /identity/v3/users/ca102d4720334b548b089adb37f0cf91 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:39.713685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d34a7b-857b-415c-a260-3a197e448866 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:39.715367 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-a0d34a7b-857b-415c-a260-3a197e448866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:39.729307 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d34a7b-857b-415c-a260-3a197e448866 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:39.801552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b723a4c-14bd-4df4-85cc-739084753467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:39.802400 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b723a4c-14bd-4df4-85cc-739084753467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:39.802743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b723a4c-14bd-4df4-85cc-739084753467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:39.803024 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b723a4c-14bd-4df4-85cc-739084753467 admin admin] PATH_INFO: `/v3/projects/5212908648524c769045482fe4d40c69` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:39.804301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b723a4c-14bd-4df4-85cc-739084753467 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5212908648524c769045482fe4d40c69)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:39.806208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b723a4c-14bd-4df4-85cc-739084753467 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:39.908077 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-0b723a4c-14bd-4df4-85cc-739084753467 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5212908648524c769045482fe4d40c69', 'request_id': 'req-0b723a4c-14bd-4df4-85cc-739084753467'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:39.998363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-0b723a4c-14bd-4df4-85cc-739084753467 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5212908648524c769045482fe4d40c69', 'request_id': 'req-0b723a4c-14bd-4df4-85cc-739084753467'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:40.005439 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-0b723a4c-14bd-4df4-85cc-739084753467 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5212908648524c769045482fe4d40c69', 'request_id': 'req-0b723a4c-14bd-4df4-85cc-739084753467'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:40.017074 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-0b723a4c-14bd-4df4-85cc-739084753467 admin admin] The token cache is being invalidate because project 5212908648524c769045482fe4d40c69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:46:40.017393 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-0b723a4c-14bd-4df4-85cc-739084753467 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:40.018054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-0b723a4c-14bd-4df4-85cc-739084753467 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:40.019932 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1049/2081] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:46:39 2022] DELETE /identity/v3/projects/5212908648524c769045482fe4d40c69 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:40.024840 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1386326-e0df-4643-aab1-fb3240795749 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:40.025372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1386326-e0df-4643-aab1-fb3240795749 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:40.025466 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1386326-e0df-4643-aab1-fb3240795749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:40.044551 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-e1386326-e0df-4643-aab1-fb3240795749 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:40.045821 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-e1386326-e0df-4643-aab1-fb3240795749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:40.051259 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:40.105115 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1033/2082] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:40.113246 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7c6ff-4f2b-4e8b-ab0c-929014a3868f None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:40.113530 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7c6ff-4f2b-4e8b-ab0c-929014a3868f None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:40.113864 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7c6ff-4f2b-4e8b-ab0c-929014a3868f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:40.133757 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-0cc7c6ff-4f2b-4e8b-ab0c-929014a3868f None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:40.134979 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-0cc7c6ff-4f2b-4e8b-ab0c-929014a3868f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:40.142712 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1050/2083] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:40.147425 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1386326-e0df-4643-aab1-fb3240795749 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:40.156803 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1386326-e0df-4643-aab1-fb3240795749 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:40.161278 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1cc8b-1634-49b5-8728-b06fa2f904bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:40.162078 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1cc8b-1634-49b5-8728-b06fa2f904bd admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:40.162753 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1cc8b-1634-49b5-8728-b06fa2f904bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:40.168611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1cc8b-1634-49b5-8728-b06fa2f904bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:40.168611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f1cc8b-1634-49b5-8728-b06fa2f904bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:46:40.168611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f1cc8b-1634-49b5-8728-b06fa2f904bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:40.168611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f1cc8b-1634-49b5-8728-b06fa2f904bd admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:40.180033 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1034/2084] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:46:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:46:40.181754 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7c6ff-4f2b-4e8b-ab0c-929014a3868f None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:40.190925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7c6ff-4f2b-4e8b-ab0c-929014a3868f None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:40.196278 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf4c05-2781-4b9d-b45a-09f14e6a99de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:40.196930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf4c05-2781-4b9d-b45a-09f14e6a99de admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:40.197209 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf4c05-2781-4b9d-b45a-09f14e6a99de admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:40.197482 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf4c05-2781-4b9d-b45a-09f14e6a99de admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:40.198161 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf4c05-2781-4b9d-b45a-09f14e6a99de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:40.200323 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf4c05-2781-4b9d-b45a-09f14e6a99de admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:40.297943 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1051/2085] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:46:40 2022] POST /identity/v3/projects => generated 357 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:40.301988 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1cc8b-1634-49b5-8728-b06fa2f904bd admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:40.316103 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1cc8b-1634-49b5-8728-b06fa2f904bd admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:40.370146 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9111648-7639-4b58-9f44-4823de821fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:40.370837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9111648-7639-4b58-9f44-4823de821fd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:40.371120 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9111648-7639-4b58-9f44-4823de821fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:40.371455 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9111648-7639-4b58-9f44-4823de821fd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:40.372145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9111648-7639-4b58-9f44-4823de821fd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:40.374053 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9111648-7639-4b58-9f44-4823de821fd4 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:40.441632 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1035/2086] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:46:40 2022] POST /identity/v3/users => generated 279 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:40.447431 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf4c05-2781-4b9d-b45a-09f14e6a99de admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:40.458012 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf4c05-2781-4b9d-b45a-09f14e6a99de admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:40.462774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8f62a-fe48-4a8c-8c14-a117e181b181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:40.463469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8f62a-fe48-4a8c-8c14-a117e181b181 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:40.463740 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8f62a-fe48-4a8c-8c14-a117e181b181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:40.464015 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8f62a-fe48-4a8c-8c14-a117e181b181 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:40.464773 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b8f62a-fe48-4a8c-8c14-a117e181b181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:40.466364 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b8f62a-fe48-4a8c-8c14-a117e181b181 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:40.472267 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:40.479077 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1052/2087] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:40.483082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9111648-7639-4b58-9f44-4823de821fd4 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:40.494020 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9111648-7639-4b58-9f44-4823de821fd4 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:40.498659 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ebbc9-a5b2-4ed0-9717-ed7a040499a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:40.499301 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ebbc9-a5b2-4ed0-9717-ed7a040499a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:40.499649 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ebbc9-a5b2-4ed0-9717-ed7a040499a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:40.499906 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ebbc9-a5b2-4ed0-9717-ed7a040499a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:40.500619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ebbc9-a5b2-4ed0-9717-ed7a040499a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:40.502191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ebbc9-a5b2-4ed0-9717-ed7a040499a4 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:40.512671 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1036/2088] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:46:40.516851 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8f62a-fe48-4a8c-8c14-a117e181b181 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:40.527283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8f62a-fe48-4a8c-8c14-a117e181b181 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:40.531981 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719d673-0269-41ac-bca5-0b0e38a2d4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:40.532845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719d673-0269-41ac-bca5-0b0e38a2d4af admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:40.533127 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719d673-0269-41ac-bca5-0b0e38a2d4af admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:40.533400 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719d673-0269-41ac-bca5-0b0e38a2d4af admin admin] PATH_INFO: `/v3/projects/b53cd052002448e1bbb54e2b99271802/users/56d651e2aa7546f4ac7aec7150c119d3/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:40.549017 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719d673-0269-41ac-bca5-0b0e38a2d4af admin admin] RBAC: Authorizing `identity:create_grant(project_id=b53cd052002448e1bbb54e2b99271802, user_id=56d651e2aa7546f4ac7aec7150c119d3, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:40.551864 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719d673-0269-41ac-bca5-0b0e38a2d4af admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:40.619429 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1053/2089] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:40 2022] PUT /identity/v3/projects/b53cd052002448e1bbb54e2b99271802/users/56d651e2aa7546f4ac7aec7150c119d3/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:40.630598 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bb9b5-30d8-4e77-9474-b8c259177568 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:40.630886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bb9b5-30d8-4e77-9474-b8c259177568 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:40.631164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bb9b5-30d8-4e77-9474-b8c259177568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:40.659438 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-594bb9b5-30d8-4e77-9474-b8c259177568 None None] MFA Rules not processed for user `56d651e2aa7546f4ac7aec7150c119d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:40.660973 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-594bb9b5-30d8-4e77-9474-b8c259177568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:40.727590 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1037/2090] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:40 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:40.732353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719d673-0269-41ac-bca5-0b0e38a2d4af admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:40.798375 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719d673-0269-41ac-bca5-0b0e38a2d4af admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:40.805142 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ebea9-034e-49e8-8f71-825661771e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:40.805801 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ebea9-034e-49e8-8f71-825661771e1c admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:40.806108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ebea9-034e-49e8-8f71-825661771e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:40.806351 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ebea9-034e-49e8-8f71-825661771e1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:40.807070 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ebea9-034e-49e8-8f71-825661771e1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:40.809059 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ebea9-034e-49e8-8f71-825661771e1c admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:40.920732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1054/2091] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:46:40 2022] POST /identity/v3/projects => generated 357 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:40.924761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bb9b5-30d8-4e77-9474-b8c259177568 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:40.936794 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bb9b5-30d8-4e77-9474-b8c259177568 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:40.940298 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:40.995812 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d5e88-469a-414b-adda-9aef7d087bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:40.996474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d5e88-469a-414b-adda-9aef7d087bad admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:40.996758 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d5e88-469a-414b-adda-9aef7d087bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:40.997036 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d5e88-469a-414b-adda-9aef7d087bad admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:40.997711 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d5e88-469a-414b-adda-9aef7d087bad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:40.999654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d5e88-469a-414b-adda-9aef7d087bad admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:41.116713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1038/2092] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:46:40 2022] POST /identity/v3/users => generated 285 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:41.117959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ebea9-034e-49e8-8f71-825661771e1c admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:41.128480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ebea9-034e-49e8-8f71-825661771e1c admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:41.134314 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3f3bf-3a84-415e-87fb-4fd0b164fe1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:41.135391 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3f3bf-3a84-415e-87fb-4fd0b164fe1e admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:41.135771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3f3bf-3a84-415e-87fb-4fd0b164fe1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:41.136109 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3f3bf-3a84-415e-87fb-4fd0b164fe1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:41.136988 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3f3bf-3a84-415e-87fb-4fd0b164fe1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:41.138831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3f3bf-3a84-415e-87fb-4fd0b164fe1e admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:41.150758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1055/2093] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:41.154878 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d5e88-469a-414b-adda-9aef7d087bad admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:41.165605 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d5e88-469a-414b-adda-9aef7d087bad admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:41.171333 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfac2e-d894-4be6-b03c-37e1eda2d3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:41.172355 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfac2e-d894-4be6-b03c-37e1eda2d3ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:41.172686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfac2e-d894-4be6-b03c-37e1eda2d3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:41.172993 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfac2e-d894-4be6-b03c-37e1eda2d3ab admin admin] PATH_INFO: `/v3/projects/1fccda09b59148819bedc73cc32bdff3/users/d5c1215931f5413799f4c66171df330f/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:41.188965 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfac2e-d894-4be6-b03c-37e1eda2d3ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fccda09b59148819bedc73cc32bdff3, user_id=d5c1215931f5413799f4c66171df330f, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:41.191835 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfac2e-d894-4be6-b03c-37e1eda2d3ab admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:41.314568 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1039/2094] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:41 2022] PUT /identity/v3/projects/1fccda09b59148819bedc73cc32bdff3/users/d5c1215931f5413799f4c66171df330f/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:41.317958 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3f3bf-3a84-415e-87fb-4fd0b164fe1e admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:41.329939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3f3bf-3a84-415e-87fb-4fd0b164fe1e admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:41.392508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d988aa-b2e8-4782-a100-329e171aa3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:41.393166 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d988aa-b2e8-4782-a100-329e171aa3c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:41.393468 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d988aa-b2e8-4782-a100-329e171aa3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:41.394136 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d988aa-b2e8-4782-a100-329e171aa3c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:41.394561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d988aa-b2e8-4782-a100-329e171aa3c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:41.396181 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d988aa-b2e8-4782-a100-329e171aa3c8 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:41.407173 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1056/2095] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:41.411376 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfac2e-d894-4be6-b03c-37e1eda2d3ab admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:41.423499 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfac2e-d894-4be6-b03c-37e1eda2d3ab admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:41.429013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f7557-5d70-4b1f-b7d7-95c088107bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:41.429995 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f7557-5d70-4b1f-b7d7-95c088107bed admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:41.430355 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f7557-5d70-4b1f-b7d7-95c088107bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:41.430725 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f7557-5d70-4b1f-b7d7-95c088107bed admin admin] PATH_INFO: `/v3/projects/1fccda09b59148819bedc73cc32bdff3/users/d5c1215931f5413799f4c66171df330f/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:41.432895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f7557-5d70-4b1f-b7d7-95c088107bed admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fccda09b59148819bedc73cc32bdff3, user_id=d5c1215931f5413799f4c66171df330f, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:41.435737 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f7557-5d70-4b1f-b7d7-95c088107bed admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:41.447358 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1040/2096] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:41 2022] PUT /identity/v3/projects/1fccda09b59148819bedc73cc32bdff3/users/d5c1215931f5413799f4c66171df330f/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:41.457273 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2aa71-3a78-4ab0-a12c-e6125f001147 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:41.457560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2aa71-3a78-4ab0-a12c-e6125f001147 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:41.457891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2aa71-3a78-4ab0-a12c-e6125f001147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:41.484579 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-f3b2aa71-3a78-4ab0-a12c-e6125f001147 None None] MFA Rules not processed for user `d5c1215931f5413799f4c66171df330f`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:41.485762 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-f3b2aa71-3a78-4ab0-a12c-e6125f001147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:41.519731 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:41.573913 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1057/2097] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:41 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:41.633917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f7557-5d70-4b1f-b7d7-95c088107bed admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:41.638654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-4f4f7557-5d70-4b1f-b7d7-95c088107bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:41.739207 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f7557-5d70-4b1f-b7d7-95c088107bed admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:41.749604 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50522539-f0b7-42c7-be5e-8ab17c1f5ab4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:41.750958 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50522539-f0b7-42c7-be5e-8ab17c1f5ab4 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:41.751840 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50522539-f0b7-42c7-be5e-8ab17c1f5ab4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:41.752123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50522539-f0b7-42c7-be5e-8ab17c1f5ab4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:41.768852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50522539-f0b7-42c7-be5e-8ab17c1f5ab4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:41.770964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50522539-f0b7-42c7-be5e-8ab17c1f5ab4 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:41.783658 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1041/2098] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:41 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:42.080810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2aa71-3a78-4ab0-a12c-e6125f001147 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:42.092696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2aa71-3a78-4ab0-a12c-e6125f001147 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:42.096998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3603f8c-fb8a-4f03-969a-75fac4e270a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:42.097713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3603f8c-fb8a-4f03-969a-75fac4e270a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:42.098064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3603f8c-fb8a-4f03-969a-75fac4e270a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:42.098439 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3603f8c-fb8a-4f03-969a-75fac4e270a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:42.160985 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3603f8c-fb8a-4f03-969a-75fac4e270a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:42.162574 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3603f8c-fb8a-4f03-969a-75fac4e270a6 service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:42.171768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1058/2099] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:42 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:42.847043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50522539-f0b7-42c7-be5e-8ab17c1f5ab4 service cinder] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:42.849189 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-50522539-f0b7-42c7-be5e-8ab17c1f5ab4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:42.965248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50522539-f0b7-42c7-be5e-8ab17c1f5ab4 service cinder] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:42.986891 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c7f25-026f-4cb1-b54b-e79268a27357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:42.987797 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c7f25-026f-4cb1-b54b-e79268a27357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:42.988142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c7f25-026f-4cb1-b54b-e79268a27357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:42.988426 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c7f25-026f-4cb1-b54b-e79268a27357 admin admin] PATH_INFO: `/v3/users/686baef51fc54092a9b7d16b6d8474c8` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:42.992658 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c7f25-026f-4cb1-b54b-e79268a27357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=686baef51fc54092a9b7d16b6d8474c8)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:42.994816 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c7f25-026f-4cb1-b54b-e79268a27357 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:43.003900 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:43.184527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3603f8c-fb8a-4f03-969a-75fac4e270a6 service cinder] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:43.186622 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c3603f8c-fb8a-4f03-969a-75fac4e270a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:43.201578 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3603f8c-fb8a-4f03-969a-75fac4e270a6 service cinder] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:43.215453 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e2e67-6330-426e-88af-2a33ed85d80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:43.216371 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e2e67-6330-426e-88af-2a33ed85d80f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:43.216655 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e2e67-6330-426e-88af-2a33ed85d80f admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:43.216934 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e2e67-6330-426e-88af-2a33ed85d80f admin admin] PATH_INFO: `/v3/users/5d8f36b08f4a4c97b9deb9afde57fe7a` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:43.218226 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e2e67-6330-426e-88af-2a33ed85d80f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d8f36b08f4a4c97b9deb9afde57fe7a)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:43.220496 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e2e67-6330-426e-88af-2a33ed85d80f admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:43.260836 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-173c7f25-026f-4cb1-b54b-e79268a27357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '686baef51fc54092a9b7d16b6d8474c8'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:43.269937 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-173c7f25-026f-4cb1-b54b-e79268a27357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '686baef51fc54092a9b7d16b6d8474c8'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:43.270704 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-173c7f25-026f-4cb1-b54b-e79268a27357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '686baef51fc54092a9b7d16b6d8474c8'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:43.284211 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-173c7f25-026f-4cb1-b54b-e79268a27357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '686baef51fc54092a9b7d16b6d8474c8'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:43.378524 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-173c7f25-026f-4cb1-b54b-e79268a27357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '686baef51fc54092a9b7d16b6d8474c8'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:43.390453 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1042/2100] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:46:42 2022] DELETE /identity/v3/users/686baef51fc54092a9b7d16b6d8474c8 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:43.394950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8804f25-691a-4c44-92b8-6c9f9a814897 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:43.395244 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8804f25-691a-4c44-92b8-6c9f9a814897 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:43.395788 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8804f25-691a-4c44-92b8-6c9f9a814897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:43.414515 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-d8804f25-691a-4c44-92b8-6c9f9a814897 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:43.415739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-d8804f25-691a-4c44-92b8-6c9f9a814897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:43.485973 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1043/2101] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:46:43.493533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8804f25-691a-4c44-92b8-6c9f9a814897 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:43.495677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-d8804f25-691a-4c44-92b8-6c9f9a814897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:43.530989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-fb4e2e67-6330-426e-88af-2a33ed85d80f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d8f36b08f4a4c97b9deb9afde57fe7a'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:43.541686 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-fb4e2e67-6330-426e-88af-2a33ed85d80f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d8f36b08f4a4c97b9deb9afde57fe7a'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:43.549696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-fb4e2e67-6330-426e-88af-2a33ed85d80f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d8f36b08f4a4c97b9deb9afde57fe7a'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:43.549696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-fb4e2e67-6330-426e-88af-2a33ed85d80f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d8f36b08f4a4c97b9deb9afde57fe7a'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:43.561459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-fb4e2e67-6330-426e-88af-2a33ed85d80f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d8f36b08f4a4c97b9deb9afde57fe7a'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:43.574185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8804f25-691a-4c44-92b8-6c9f9a814897 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:43.580797 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb43f8-81e9-4640-abd0-36f66747ed79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:43.581470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb43f8-81e9-4640-abd0-36f66747ed79 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:43.581752 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb43f8-81e9-4640-abd0-36f66747ed79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:43.582030 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb43f8-81e9-4640-abd0-36f66747ed79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:43.584034 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-0fdb43f8-81e9-4640-abd0-36f66747ed79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:43.615059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb43f8-81e9-4640-abd0-36f66747ed79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:43.617095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb43f8-81e9-4640-abd0-36f66747ed79 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:43.631466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1059/2102] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:46:43 2022] DELETE /identity/v3/users/5d8f36b08f4a4c97b9deb9afde57fe7a => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:46:43.636697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57e934-eb72-4314-85b9-3139bdc951a3 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:43.636853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57e934-eb72-4314-85b9-3139bdc951a3 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:43.637139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57e934-eb72-4314-85b9-3139bdc951a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:43.641307 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1044/2103] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:46:43.654533 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-0b57e934-eb72-4314-85b9-3139bdc951a3 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:43.655756 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-0b57e934-eb72-4314-85b9-3139bdc951a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:43.709396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:43.742958 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1060/2104] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:46:43.754166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb43f8-81e9-4640-abd0-36f66747ed79 service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:43.755186 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-0fdb43f8-81e9-4640-abd0-36f66747ed79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:43.767676 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb43f8-81e9-4640-abd0-36f66747ed79 service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:43.792099 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:43.825795 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706cf60f-f304-4a27-b5bc-1b41488b8c42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:43.826481 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706cf60f-f304-4a27-b5bc-1b41488b8c42 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:43.826770 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706cf60f-f304-4a27-b5bc-1b41488b8c42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:43.827050 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706cf60f-f304-4a27-b5bc-1b41488b8c42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:43.837462 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706cf60f-f304-4a27-b5bc-1b41488b8c42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:43.839056 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706cf60f-f304-4a27-b5bc-1b41488b8c42 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:43.849865 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1045/2105] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:43.997913 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b57e934-eb72-4314-85b9-3139bdc951a3 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:43.999516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-0b57e934-eb72-4314-85b9-3139bdc951a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:44.016722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b57e934-eb72-4314-85b9-3139bdc951a3 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:44.022620 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384be531-bd0c-4eb1-b198-3ff90795db63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:44.024193 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384be531-bd0c-4eb1-b198-3ff90795db63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:44.024527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384be531-bd0c-4eb1-b198-3ff90795db63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:44.025190 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384be531-bd0c-4eb1-b198-3ff90795db63 admin admin] PATH_INFO: `/v3/users/b2b08a473fe64a8d964507711e21f8d7` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:44.026438 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384be531-bd0c-4eb1-b198-3ff90795db63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2b08a473fe64a8d964507711e21f8d7)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:44.028810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384be531-bd0c-4eb1-b198-3ff90795db63 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:44.282369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-384be531-bd0c-4eb1-b198-3ff90795db63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2b08a473fe64a8d964507711e21f8d7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:44.291919 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-384be531-bd0c-4eb1-b198-3ff90795db63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2b08a473fe64a8d964507711e21f8d7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:44.292885 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-384be531-bd0c-4eb1-b198-3ff90795db63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2b08a473fe64a8d964507711e21f8d7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:44.293703 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-384be531-bd0c-4eb1-b198-3ff90795db63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2b08a473fe64a8d964507711e21f8d7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:44.308957 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706cf60f-f304-4a27-b5bc-1b41488b8c42 service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:44.310696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-706cf60f-f304-4a27-b5bc-1b41488b8c42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:44.324691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-384be531-bd0c-4eb1-b198-3ff90795db63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2b08a473fe64a8d964507711e21f8d7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:44.327089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706cf60f-f304-4a27-b5bc-1b41488b8c42 service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:44.333099 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa006305-e991-4cc2-ab83-7884bdc5d091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:44.334187 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa006305-e991-4cc2-ab83-7884bdc5d091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:44.334680 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa006305-e991-4cc2-ab83-7884bdc5d091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:44.335493 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa006305-e991-4cc2-ab83-7884bdc5d091 admin admin] PATH_INFO: `/v3/projects/6457b8c69ee149ec9399da76a424d877` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:44.336889 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa006305-e991-4cc2-ab83-7884bdc5d091 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6457b8c69ee149ec9399da76a424d877)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:44.338925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa006305-e991-4cc2-ab83-7884bdc5d091 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:44.420873 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1061/2106] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:46:43 2022] DELETE /identity/v3/users/b2b08a473fe64a8d964507711e21f8d7 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:44.425124 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0bc626-626e-441e-8663-daa2621afa36 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:44.425416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0bc626-626e-441e-8663-daa2621afa36 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:44.425695 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0bc626-626e-441e-8663-daa2621afa36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:44.448300 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-0f0bc626-626e-441e-8663-daa2621afa36 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:44.450449 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-0f0bc626-626e-441e-8663-daa2621afa36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:44.459123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-aa006305-e991-4cc2-ab83-7884bdc5d091 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6457b8c69ee149ec9399da76a424d877', 'request_id': 'req-aa006305-e991-4cc2-ab83-7884bdc5d091'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:44.529489 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1062/2107] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:46:44.535827 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-aa006305-e991-4cc2-ab83-7884bdc5d091 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6457b8c69ee149ec9399da76a424d877', 'request_id': 'req-aa006305-e991-4cc2-ab83-7884bdc5d091'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:44.542960 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0bc626-626e-441e-8663-daa2621afa36 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:44.543447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-aa006305-e991-4cc2-ab83-7884bdc5d091 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6457b8c69ee149ec9399da76a424d877', 'request_id': 'req-aa006305-e991-4cc2-ab83-7884bdc5d091'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:44.544827 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-0f0bc626-626e-441e-8663-daa2621afa36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:44.566050 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-aa006305-e991-4cc2-ab83-7884bdc5d091 admin admin] The token cache is being invalidate because project 6457b8c69ee149ec9399da76a424d877 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:46:44.566663 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-aa006305-e991-4cc2-ab83-7884bdc5d091 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:44.568865 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-aa006305-e991-4cc2-ab83-7884bdc5d091 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:44.572307 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1046/2108] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:46:44 2022] DELETE /identity/v3/projects/6457b8c69ee149ec9399da76a424d877 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:44.576426 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa006305-e991-4cc2-ab83-7884bdc5d091 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:44.580114 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-aa006305-e991-4cc2-ab83-7884bdc5d091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:44.596743 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa006305-e991-4cc2-ab83-7884bdc5d091 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:44.605555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24873c52-c354-4991-86ab-055437277b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:44.607003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24873c52-c354-4991-86ab-055437277b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:44.607290 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24873c52-c354-4991-86ab-055437277b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:44.607567 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24873c52-c354-4991-86ab-055437277b23 admin admin] PATH_INFO: `/v3/users/80d8a557927d45e8a002e44d8bc8f479` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:44.608783 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24873c52-c354-4991-86ab-055437277b23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80d8a557927d45e8a002e44d8bc8f479)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:44.610861 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24873c52-c354-4991-86ab-055437277b23 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:44.615934 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0bc626-626e-441e-8663-daa2621afa36 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:44.621829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d93018-4a25-414f-866a-7637b81a3a68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:44.622557 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d93018-4a25-414f-866a-7637b81a3a68 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:44.622852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d93018-4a25-414f-866a-7637b81a3a68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:44.623181 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d93018-4a25-414f-866a-7637b81a3a68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:44.625022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c8d93018-4a25-414f-866a-7637b81a3a68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:44.639745 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d93018-4a25-414f-866a-7637b81a3a68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:44.641220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d93018-4a25-414f-866a-7637b81a3a68 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:44.651923 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:44.656395 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1063/2109] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:44.656937 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d93018-4a25-414f-866a-7637b81a3a68 service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:44.658976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c8d93018-4a25-414f-866a-7637b81a3a68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:44.677989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d93018-4a25-414f-866a-7637b81a3a68 service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:44.683683 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8880cc-1a5e-40cb-a2ec-92061b2176d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:44.684559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8880cc-1a5e-40cb-a2ec-92061b2176d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:44.684844 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8880cc-1a5e-40cb-a2ec-92061b2176d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:44.685344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8880cc-1a5e-40cb-a2ec-92061b2176d3 admin admin] PATH_INFO: `/v3/projects/5bc8701c27f94a1dbf29aa5b988e030e` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:44.686330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8880cc-1a5e-40cb-a2ec-92061b2176d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bc8701c27f94a1dbf29aa5b988e030e)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:44.688338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8880cc-1a5e-40cb-a2ec-92061b2176d3 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:44.849424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-24873c52-c354-4991-86ab-055437277b23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80d8a557927d45e8a002e44d8bc8f479'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:44.854027 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-6e8880cc-1a5e-40cb-a2ec-92061b2176d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bc8701c27f94a1dbf29aa5b988e030e', 'request_id': 'req-6e8880cc-1a5e-40cb-a2ec-92061b2176d3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:44.861425 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-24873c52-c354-4991-86ab-055437277b23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80d8a557927d45e8a002e44d8bc8f479'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:44.862332 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-24873c52-c354-4991-86ab-055437277b23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80d8a557927d45e8a002e44d8bc8f479'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:44.884739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-24873c52-c354-4991-86ab-055437277b23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80d8a557927d45e8a002e44d8bc8f479'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:44.918496 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-6e8880cc-1a5e-40cb-a2ec-92061b2176d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bc8701c27f94a1dbf29aa5b988e030e', 'request_id': 'req-6e8880cc-1a5e-40cb-a2ec-92061b2176d3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:44.939217 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-6e8880cc-1a5e-40cb-a2ec-92061b2176d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bc8701c27f94a1dbf29aa5b988e030e', 'request_id': 'req-6e8880cc-1a5e-40cb-a2ec-92061b2176d3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:44.954761 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-6e8880cc-1a5e-40cb-a2ec-92061b2176d3 admin admin] The token cache is being invalidate because project 5bc8701c27f94a1dbf29aa5b988e030e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:46:44.955055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-6e8880cc-1a5e-40cb-a2ec-92061b2176d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:44.955940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-6e8880cc-1a5e-40cb-a2ec-92061b2176d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:44.957549 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1064/2110] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:46:44 2022] DELETE /identity/v3/projects/5bc8701c27f94a1dbf29aa5b988e030e => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:44.958328 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8880cc-1a5e-40cb-a2ec-92061b2176d3 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:44.959783 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-6e8880cc-1a5e-40cb-a2ec-92061b2176d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:45.012623 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-24873c52-c354-4991-86ab-055437277b23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80d8a557927d45e8a002e44d8bc8f479'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:45.020342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1047/2111] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:46:44 2022] DELETE /identity/v3/users/80d8a557927d45e8a002e44d8bc8f479 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:45.022058 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c0a62-c324-4054-8033-1bd2da9dd90b None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:45.022981 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c0a62-c324-4054-8033-1bd2da9dd90b None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:45.023841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c0a62-c324-4054-8033-1bd2da9dd90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:45.045172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-481c0a62-c324-4054-8033-1bd2da9dd90b None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:45.046595 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-481c0a62-c324-4054-8033-1bd2da9dd90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:45.061092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8880cc-1a5e-40cb-a2ec-92061b2176d3 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:45.072986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:45.140532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1048/2112] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:45.143071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71706e06-0b92-4ba0-a400-c9d0ed4dc5c3 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:45.144057 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71706e06-0b92-4ba0-a400-c9d0ed4dc5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:45.144489 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71706e06-0b92-4ba0-a400-c9d0ed4dc5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:45.165426 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-71706e06-0b92-4ba0-a400-c9d0ed4dc5c3 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:45.167238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-71706e06-0b92-4ba0-a400-c9d0ed4dc5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:45.179951 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406196b3-e985-4802-a64d-370ba3ea4150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:45.181157 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1049/2113] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:45.181473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406196b3-e985-4802-a64d-370ba3ea4150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:45.182077 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71706e06-0b92-4ba0-a400-c9d0ed4dc5c3 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:45.182702 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406196b3-e985-4802-a64d-370ba3ea4150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:45.183039 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406196b3-e985-4802-a64d-370ba3ea4150 admin admin] PATH_INFO: `/v3/users/0fc43db8323043a0a066f2be1eb2a9f6` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:45.184094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-71706e06-0b92-4ba0-a400-c9d0ed4dc5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:45.191504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406196b3-e985-4802-a64d-370ba3ea4150 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fc43db8323043a0a066f2be1eb2a9f6)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:45.195138 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406196b3-e985-4802-a64d-370ba3ea4150 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:45.201754 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71706e06-0b92-4ba0-a400-c9d0ed4dc5c3 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:45.210642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9594ae-0a5b-431d-996f-688cc509b1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:45.213066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9594ae-0a5b-431d-996f-688cc509b1e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:45.213559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9594ae-0a5b-431d-996f-688cc509b1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:45.213862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9594ae-0a5b-431d-996f-688cc509b1e3 admin admin] PATH_INFO: `/v3/projects/ea3a07549a8c41eeb4604f65d39ae33e` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:45.215254 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9594ae-0a5b-431d-996f-688cc509b1e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea3a07549a8c41eeb4604f65d39ae33e)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:45.218054 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9594ae-0a5b-431d-996f-688cc509b1e3 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:45.512166 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-406196b3-e985-4802-a64d-370ba3ea4150 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fc43db8323043a0a066f2be1eb2a9f6'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:45.512708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-6d9594ae-0a5b-431d-996f-688cc509b1e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea3a07549a8c41eeb4604f65d39ae33e', 'request_id': 'req-6d9594ae-0a5b-431d-996f-688cc509b1e3'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:45.519297 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-406196b3-e985-4802-a64d-370ba3ea4150 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fc43db8323043a0a066f2be1eb2a9f6'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:45.519998 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-406196b3-e985-4802-a64d-370ba3ea4150 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fc43db8323043a0a066f2be1eb2a9f6'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:45.520624 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-406196b3-e985-4802-a64d-370ba3ea4150 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fc43db8323043a0a066f2be1eb2a9f6'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:45.533901 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-406196b3-e985-4802-a64d-370ba3ea4150 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fc43db8323043a0a066f2be1eb2a9f6'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:45.602841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-6d9594ae-0a5b-431d-996f-688cc509b1e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea3a07549a8c41eeb4604f65d39ae33e', 'request_id': 'req-6d9594ae-0a5b-431d-996f-688cc509b1e3'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:45.619601 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-6d9594ae-0a5b-431d-996f-688cc509b1e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea3a07549a8c41eeb4604f65d39ae33e', 'request_id': 'req-6d9594ae-0a5b-431d-996f-688cc509b1e3'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:45.651732 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-6d9594ae-0a5b-431d-996f-688cc509b1e3 admin admin] The token cache is being invalidate because project ea3a07549a8c41eeb4604f65d39ae33e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:46:45.652577 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-6d9594ae-0a5b-431d-996f-688cc509b1e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:45.653358 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-6d9594ae-0a5b-431d-996f-688cc509b1e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:45.664578 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1050/2114] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:46:45 2022] DELETE /identity/v3/projects/ea3a07549a8c41eeb4604f65d39ae33e => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:45.668107 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9594ae-0a5b-431d-996f-688cc509b1e3 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:45.672449 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-6d9594ae-0a5b-431d-996f-688cc509b1e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:45.727135 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1065/2115] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:46:44 2022] DELETE /identity/v3/users/0fc43db8323043a0a066f2be1eb2a9f6 => generated 0 bytes in 764 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:45.728984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fddb7c-a2f4-4990-a76a-67a8f39a2afe None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:45.733680 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fddb7c-a2f4-4990-a76a-67a8f39a2afe None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:45.734695 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fddb7c-a2f4-4990-a76a-67a8f39a2afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:45.807529 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-c8fddb7c-a2f4-4990-a76a-67a8f39a2afe None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:45.808793 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c8fddb7c-a2f4-4990-a76a-67a8f39a2afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:45.887835 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9594ae-0a5b-431d-996f-688cc509b1e3 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:45.952581 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1066/2116] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:45.953820 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fddb7c-a2f4-4990-a76a-67a8f39a2afe None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:45.956271 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c8fddb7c-a2f4-4990-a76a-67a8f39a2afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:45.975761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec174bd1-a55c-4259-af4b-8bcb47e12346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:45.975875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fddb7c-a2f4-4990-a76a-67a8f39a2afe None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:45.975953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec174bd1-a55c-4259-af4b-8bcb47e12346 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:45.975953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec174bd1-a55c-4259-af4b-8bcb47e12346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:45.976499 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec174bd1-a55c-4259-af4b-8bcb47e12346 admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:45.978568 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec174bd1-a55c-4259-af4b-8bcb47e12346 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:46:45.978691 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec174bd1-a55c-4259-af4b-8bcb47e12346 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:45.982591 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec174bd1-a55c-4259-af4b-8bcb47e12346 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:45.982918 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a94247-14ea-4a45-b463-d81f213968ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:45.984225 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a94247-14ea-4a45-b463-d81f213968ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:45.984902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a94247-14ea-4a45-b463-d81f213968ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:45.985328 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a94247-14ea-4a45-b463-d81f213968ef admin admin] PATH_INFO: `/v3/projects/b64091c38ed04759a2b6bc51ff0e56cb` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:45.987684 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a94247-14ea-4a45-b463-d81f213968ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=b64091c38ed04759a2b6bc51ff0e56cb)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:45.990524 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a94247-14ea-4a45-b463-d81f213968ef admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:45.996396 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1051/2117] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:46:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:46:45.997272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec174bd1-a55c-4259-af4b-8bcb47e12346 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:46.003865 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-ec174bd1-a55c-4259-af4b-8bcb47e12346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:46.041293 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:46.070353 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec174bd1-a55c-4259-af4b-8bcb47e12346 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:46.075220 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b4de6-7654-4f40-ad05-0cf00a2c3ffb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:46.075871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b4de6-7654-4f40-ad05-0cf00a2c3ffb service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:46.076161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b4de6-7654-4f40-ad05-0cf00a2c3ffb service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:46.076432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b4de6-7654-4f40-ad05-0cf00a2c3ffb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:46.086444 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b4de6-7654-4f40-ad05-0cf00a2c3ffb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:46.087885 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b4de6-7654-4f40-ad05-0cf00a2c3ffb service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:46.098362 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1052/2118] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:46.099221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b4de6-7654-4f40-ad05-0cf00a2c3ffb service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:46.110936 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b4de6-7654-4f40-ad05-0cf00a2c3ffb service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:46.134254 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f6a94247-14ea-4a45-b463-d81f213968ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b64091c38ed04759a2b6bc51ff0e56cb', 'request_id': 'req-f6a94247-14ea-4a45-b463-d81f213968ef'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:46.169672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfde35b-125a-4445-ad41-739a59046dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:46.170746 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfde35b-125a-4445-ad41-739a59046dd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:46.170746 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfde35b-125a-4445-ad41-739a59046dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:46.170955 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfde35b-125a-4445-ad41-739a59046dd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:46.171588 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfde35b-125a-4445-ad41-739a59046dd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:46.173509 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfde35b-125a-4445-ad41-739a59046dd3 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:46.207652 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f6a94247-14ea-4a45-b463-d81f213968ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b64091c38ed04759a2b6bc51ff0e56cb', 'request_id': 'req-f6a94247-14ea-4a45-b463-d81f213968ef'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:46.216311 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f6a94247-14ea-4a45-b463-d81f213968ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b64091c38ed04759a2b6bc51ff0e56cb', 'request_id': 'req-f6a94247-14ea-4a45-b463-d81f213968ef'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:46.238440 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f6a94247-14ea-4a45-b463-d81f213968ef admin admin] The token cache is being invalidate because project b64091c38ed04759a2b6bc51ff0e56cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:46:46.238670 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f6a94247-14ea-4a45-b463-d81f213968ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:46.242598 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f6a94247-14ea-4a45-b463-d81f213968ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:46.244483 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1067/2119] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:46:45 2022] DELETE /identity/v3/projects/b64091c38ed04759a2b6bc51ff0e56cb => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:46.248820 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a94247-14ea-4a45-b463-d81f213968ef admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:46.251313 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-f6a94247-14ea-4a45-b463-d81f213968ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:46.274245 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1053/2120] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:46:46 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:46.278582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a94247-14ea-4a45-b463-d81f213968ef admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:46.283895 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfde35b-125a-4445-ad41-739a59046dd3 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:46.287363 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-acfde35b-125a-4445-ad41-739a59046dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:46.358962 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d101e-0b47-417c-aeb8-3cd40561548d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:46.359903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d101e-0b47-417c-aeb8-3cd40561548d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:46.360190 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d101e-0b47-417c-aeb8-3cd40561548d admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:46.360468 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d101e-0b47-417c-aeb8-3cd40561548d admin admin] PATH_INFO: `/v3/users/b04ebd2f125e4eae9ba4334c18f36797` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:46.361861 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d101e-0b47-417c-aeb8-3cd40561548d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b04ebd2f125e4eae9ba4334c18f36797)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:46.363873 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d101e-0b47-417c-aeb8-3cd40561548d admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:46.396210 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfde35b-125a-4445-ad41-739a59046dd3 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:46.396210 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747dead6-e1d4-4ccb-8192-31007b0432ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:46.396210 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dead6-e1d4-4ccb-8192-31007b0432ee admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:46.396210 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dead6-e1d4-4ccb-8192-31007b0432ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:46.397524 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dead6-e1d4-4ccb-8192-31007b0432ee admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:46.397524 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747dead6-e1d4-4ccb-8192-31007b0432ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:46.397524 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747dead6-e1d4-4ccb-8192-31007b0432ee admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:46.678273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1054/2121] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:46:46 2022] POST /identity/v3/users => generated 292 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:46.680384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b5204-ba5c-4262-859f-76b9045e70c9 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:46.680681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b5204-ba5c-4262-859f-76b9045e70c9 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:46.680961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b5204-ba5c-4262-859f-76b9045e70c9 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:46.682226 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1055/2122] 10.222.0.15 () {54 vars in 921 bytes} [Mon Jun 27 18:46:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:46:46.682955 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b5204-ba5c-4262-859f-76b9045e70c9 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:46.687693 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-a69b5204-ba5c-4262-859f-76b9045e70c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:46.713163 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b5204-ba5c-4262-859f-76b9045e70c9 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:46.718621 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a994ab5-a659-4202-8711-f2702c16e690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:46.719709 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a994ab5-a659-4202-8711-f2702c16e690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:46.720086 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a994ab5-a659-4202-8711-f2702c16e690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:46.720086 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a994ab5-a659-4202-8711-f2702c16e690 admin admin] PATH_INFO: `/v3/projects/753b5d2157944073b351fe3eb6b0b622` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:46.721230 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a994ab5-a659-4202-8711-f2702c16e690 admin admin] RBAC: Authorizing `identity:delete_project(project_id=753b5d2157944073b351fe3eb6b0b622)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:46.723180 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a994ab5-a659-4202-8711-f2702c16e690 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:46.730257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-906d101e-0b47-417c-aeb8-3cd40561548d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b04ebd2f125e4eae9ba4334c18f36797'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:46.739888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-906d101e-0b47-417c-aeb8-3cd40561548d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b04ebd2f125e4eae9ba4334c18f36797'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:46.746505 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-906d101e-0b47-417c-aeb8-3cd40561548d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b04ebd2f125e4eae9ba4334c18f36797'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:46.746505 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-906d101e-0b47-417c-aeb8-3cd40561548d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b04ebd2f125e4eae9ba4334c18f36797'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:46.756130 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-906d101e-0b47-417c-aeb8-3cd40561548d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b04ebd2f125e4eae9ba4334c18f36797'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:46.827975 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1068/2123] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:46:46 2022] DELETE /identity/v3/users/b04ebd2f125e4eae9ba4334c18f36797 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:46:46.827975 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d101e-0b47-417c-aeb8-3cd40561548d admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:46.827975 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-906d101e-0b47-417c-aeb8-3cd40561548d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:46.844181 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-0a994ab5-a659-4202-8711-f2702c16e690 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '753b5d2157944073b351fe3eb6b0b622', 'request_id': 'req-0a994ab5-a659-4202-8711-f2702c16e690'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:46.905143 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d101e-0b47-417c-aeb8-3cd40561548d admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:46.905877 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:46.910318 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e6d9a-f56d-4aac-a798-e2516c178a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:46.911044 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e6d9a-f56d-4aac-a798-e2516c178a60 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:46.911331 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e6d9a-f56d-4aac-a798-e2516c178a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:46.911833 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e6d9a-f56d-4aac-a798-e2516c178a60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:46.912416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e6d9a-f56d-4aac-a798-e2516c178a60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:46.914709 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e6d9a-f56d-4aac-a798-e2516c178a60 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:46.926806 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1069/2124] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:46.927784 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e6d9a-f56d-4aac-a798-e2516c178a60 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:46.928280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-0a994ab5-a659-4202-8711-f2702c16e690 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '753b5d2157944073b351fe3eb6b0b622', 'request_id': 'req-0a994ab5-a659-4202-8711-f2702c16e690'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:46.930221 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-1f3e6d9a-f56d-4aac-a798-e2516c178a60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:46.941732 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-0a994ab5-a659-4202-8711-f2702c16e690 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '753b5d2157944073b351fe3eb6b0b622', 'request_id': 'req-0a994ab5-a659-4202-8711-f2702c16e690'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:46.945036 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e6d9a-f56d-4aac-a798-e2516c178a60 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:46.950319 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b690d5-cc8a-4879-8bf4-8945f5c5b7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:46.951341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b690d5-cc8a-4879-8bf4-8945f5c5b7a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:46.951623 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b690d5-cc8a-4879-8bf4-8945f5c5b7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:46.952177 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b690d5-cc8a-4879-8bf4-8945f5c5b7a0 admin admin] PATH_INFO: `/v3/users/daef56aadd1c4edfa6eeadea63050d37` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:46.953171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b690d5-cc8a-4879-8bf4-8945f5c5b7a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=daef56aadd1c4edfa6eeadea63050d37)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:46.955160 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b690d5-cc8a-4879-8bf4-8945f5c5b7a0 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:46.955785 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-0a994ab5-a659-4202-8711-f2702c16e690 admin admin] The token cache is being invalidate because project 753b5d2157944073b351fe3eb6b0b622 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:46:46.956121 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-0a994ab5-a659-4202-8711-f2702c16e690 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:46.957018 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-0a994ab5-a659-4202-8711-f2702c16e690 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:46.958868 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1056/2125] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:46:46 2022] DELETE /identity/v3/projects/753b5d2157944073b351fe3eb6b0b622 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:46.960979 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88423bfb-f6de-4377-9dfc-e4bb8a30f2b5 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:46.961420 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88423bfb-f6de-4377-9dfc-e4bb8a30f2b5 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:46.961830 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88423bfb-f6de-4377-9dfc-e4bb8a30f2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:46.990113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-88423bfb-f6de-4377-9dfc-e4bb8a30f2b5 None None] MFA Rules not processed for user `3cd6f0a1286243e4add692db01028205`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:46.991611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-88423bfb-f6de-4377-9dfc-e4bb8a30f2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:47.067073 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1057/2126] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Jun 27 18:46:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:47.068793 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88423bfb-f6de-4377-9dfc-e4bb8a30f2b5 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:47.070823 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-88423bfb-f6de-4377-9dfc-e4bb8a30f2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:47.084668 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88423bfb-f6de-4377-9dfc-e4bb8a30f2b5 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:47.089702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e481e-8ef0-41e0-b693-dae9bef58d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:47.090370 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e481e-8ef0-41e0-b693-dae9bef58d98 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:47.090707 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e481e-8ef0-41e0-b693-dae9bef58d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:47.090982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e481e-8ef0-41e0-b693-dae9bef58d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:47.091715 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21e481e-8ef0-41e0-b693-dae9bef58d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:47.093301 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21e481e-8ef0-41e0-b693-dae9bef58d98 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:47.104156 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1058/2127] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:47.105551 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e481e-8ef0-41e0-b693-dae9bef58d98 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:47.108531 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-a21e481e-8ef0-41e0-b693-dae9bef58d98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:47.123666 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e481e-8ef0-41e0-b693-dae9bef58d98 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:47.128392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6eed7-ad04-402e-88d5-eb5fec552942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:47.129258 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6eed7-ad04-402e-88d5-eb5fec552942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:47.129544 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6eed7-ad04-402e-88d5-eb5fec552942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:47.129822 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6eed7-ad04-402e-88d5-eb5fec552942 admin admin] PATH_INFO: `/v3/users/1286d65b189745e5b49d83f6eebbd8bd` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:47.131028 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed6eed7-ad04-402e-88d5-eb5fec552942 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1286d65b189745e5b49d83f6eebbd8bd)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:47.133013 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed6eed7-ad04-402e-88d5-eb5fec552942 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:47.169273 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-38b690d5-cc8a-4879-8bf4-8945f5c5b7a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daef56aadd1c4edfa6eeadea63050d37'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:47.179177 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-38b690d5-cc8a-4879-8bf4-8945f5c5b7a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daef56aadd1c4edfa6eeadea63050d37'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:47.179177 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-38b690d5-cc8a-4879-8bf4-8945f5c5b7a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daef56aadd1c4edfa6eeadea63050d37'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:47.179177 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-38b690d5-cc8a-4879-8bf4-8945f5c5b7a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daef56aadd1c4edfa6eeadea63050d37'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:47.191134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-38b690d5-cc8a-4879-8bf4-8945f5c5b7a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daef56aadd1c4edfa6eeadea63050d37'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:47.258585 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1070/2128] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:46:46 2022] DELETE /identity/v3/users/daef56aadd1c4edfa6eeadea63050d37 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:47.267200 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b690d5-cc8a-4879-8bf4-8945f5c5b7a0 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:47.267200 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-38b690d5-cc8a-4879-8bf4-8945f5c5b7a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:47.302410 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-6ed6eed7-ad04-402e-88d5-eb5fec552942 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1286d65b189745e5b49d83f6eebbd8bd'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:47.311648 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-6ed6eed7-ad04-402e-88d5-eb5fec552942 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1286d65b189745e5b49d83f6eebbd8bd'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:47.312521 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-6ed6eed7-ad04-402e-88d5-eb5fec552942 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1286d65b189745e5b49d83f6eebbd8bd'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:47.327327 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-6ed6eed7-ad04-402e-88d5-eb5fec552942 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1286d65b189745e5b49d83f6eebbd8bd'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:47.345697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b690d5-cc8a-4879-8bf4-8945f5c5b7a0 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:47.351504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ae914-8a6a-4286-bcea-74971707933c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cd6f0a1286243e4add692db01028205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:47.354185 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ae914-8a6a-4286-bcea-74971707933c service placement] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:47.354185 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ae914-8a6a-4286-bcea-74971707933c service placement] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:47.354185 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ae914-8a6a-4286-bcea-74971707933c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:47.354486 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-0b7ae914-8a6a-4286-bcea-74971707933c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:47.366674 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ae914-8a6a-4286-bcea-74971707933c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:47.368269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ae914-8a6a-4286-bcea-74971707933c service placement] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:47.378866 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1071/2129] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:47.387419 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ae914-8a6a-4286-bcea-74971707933c service placement] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:47.387419 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-0b7ae914-8a6a-4286-bcea-74971707933c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:47.406607 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-6ed6eed7-ad04-402e-88d5-eb5fec552942 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1286d65b189745e5b49d83f6eebbd8bd'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:47.419423 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1059/2130] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:46:47 2022] DELETE /identity/v3/users/1286d65b189745e5b49d83f6eebbd8bd => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:47.431778 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6eed7-ad04-402e-88d5-eb5fec552942 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:47.447261 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-6ed6eed7-ad04-402e-88d5-eb5fec552942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:47.454481 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ae914-8a6a-4286-bcea-74971707933c service placement] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:47.517206 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0edd6-afb1-4f05-ba00-476d1361b871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:47.518440 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0edd6-afb1-4f05-ba00-476d1361b871 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:47.519259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0edd6-afb1-4f05-ba00-476d1361b871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:47.519707 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0edd6-afb1-4f05-ba00-476d1361b871 admin admin] PATH_INFO: `/v3/projects/e478db03382f437fb83d6b1e74441419/users/8ae868193ad64b638e2e2a1cc1371980/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:47.520061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6eed7-ad04-402e-88d5-eb5fec552942 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:47.526155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f0e92-5b70-4d7e-9bfe-e1f632f47a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:47.527072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f0e92-5b70-4d7e-9bfe-e1f632f47a28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:47.531301 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f0e92-5b70-4d7e-9bfe-e1f632f47a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:47.531301 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f0e92-5b70-4d7e-9bfe-e1f632f47a28 admin admin] PATH_INFO: `/v3/projects/e8d467159fde4b81b8af329991d50528` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:47.531301 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f0e92-5b70-4d7e-9bfe-e1f632f47a28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8d467159fde4b81b8af329991d50528)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:47.531301 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f0e92-5b70-4d7e-9bfe-e1f632f47a28 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:47.535623 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e0edd6-afb1-4f05-ba00-476d1361b871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e478db03382f437fb83d6b1e74441419, user_id=8ae868193ad64b638e2e2a1cc1371980, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:47.538582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e0edd6-afb1-4f05-ba00-476d1361b871 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:47.596974 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:47.606752 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1072/2131] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:47 2022] PUT /identity/v3/projects/e478db03382f437fb83d6b1e74441419/users/8ae868193ad64b638e2e2a1cc1371980/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:47.607893 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0edd6-afb1-4f05-ba00-476d1361b871 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:47.618093 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-f6e0edd6-afb1-4f05-ba00-476d1361b871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:47.624158 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-9d2f0e92-5b70-4d7e-9bfe-e1f632f47a28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8d467159fde4b81b8af329991d50528', 'request_id': 'req-9d2f0e92-5b70-4d7e-9bfe-e1f632f47a28'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:47.687074 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-9d2f0e92-5b70-4d7e-9bfe-e1f632f47a28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8d467159fde4b81b8af329991d50528', 'request_id': 'req-9d2f0e92-5b70-4d7e-9bfe-e1f632f47a28'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:47.696298 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-9d2f0e92-5b70-4d7e-9bfe-e1f632f47a28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8d467159fde4b81b8af329991d50528', 'request_id': 'req-9d2f0e92-5b70-4d7e-9bfe-e1f632f47a28'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:47.698572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0edd6-afb1-4f05-ba00-476d1361b871 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:47.709176 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-9d2f0e92-5b70-4d7e-9bfe-e1f632f47a28 admin admin] The token cache is being invalidate because project e8d467159fde4b81b8af329991d50528 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:46:47.711063 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-9d2f0e92-5b70-4d7e-9bfe-e1f632f47a28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:47.711765 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-9d2f0e92-5b70-4d7e-9bfe-e1f632f47a28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:47.717791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1060/2132] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:46:47 2022] DELETE /identity/v3/projects/e8d467159fde4b81b8af329991d50528 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:47.717791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41127cae-908f-4f77-bf7c-02360340d6a3 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:47.717791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41127cae-908f-4f77-bf7c-02360340d6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:47.717791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41127cae-908f-4f77-bf7c-02360340d6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:47.758672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-41127cae-908f-4f77-bf7c-02360340d6a3 None None] MFA Rules not processed for user `8ae868193ad64b638e2e2a1cc1371980`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:47.760056 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-41127cae-908f-4f77-bf7c-02360340d6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:47.765416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d183a-eff1-459b-b09f-6f868e9dcc61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:47.766282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d183a-eff1-459b-b09f-6f868e9dcc61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:47.766596 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d183a-eff1-459b-b09f-6f868e9dcc61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:47.766873 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d183a-eff1-459b-b09f-6f868e9dcc61 admin admin] PATH_INFO: `/v3/projects/f3a70eb47fed4e959fc51400ddc3caf1` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:47.768094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d183a-eff1-459b-b09f-6f868e9dcc61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3a70eb47fed4e959fc51400ddc3caf1)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:47.770009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d183a-eff1-459b-b09f-6f868e9dcc61 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:47.828825 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1061/2133] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:47.830018 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e7c78-0f86-4970-b6af-e5c991e01f87 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:47.830307 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e7c78-0f86-4970-b6af-e5c991e01f87 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:47.830621 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e7c78-0f86-4970-b6af-e5c991e01f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:47.849529 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-484e7c78-0f86-4970-b6af-e5c991e01f87 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:47.853158 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-484e7c78-0f86-4970-b6af-e5c991e01f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:47.859181 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1062/2134] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:47.860065 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e7c78-0f86-4970-b6af-e5c991e01f87 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:47.861253 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-484e7c78-0f86-4970-b6af-e5c991e01f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:47.922449 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-233d183a-eff1-459b-b09f-6f868e9dcc61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3a70eb47fed4e959fc51400ddc3caf1', 'request_id': 'req-233d183a-eff1-459b-b09f-6f868e9dcc61'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:47.922759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e7c78-0f86-4970-b6af-e5c991e01f87 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:47.970053 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359caec-06ec-493c-99b4-e3490ca9adaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:47.970719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359caec-06ec-493c-99b4-e3490ca9adaf service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:47.971003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359caec-06ec-493c-99b4-e3490ca9adaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:47.971275 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359caec-06ec-493c-99b4-e3490ca9adaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:48.024086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-233d183a-eff1-459b-b09f-6f868e9dcc61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3a70eb47fed4e959fc51400ddc3caf1', 'request_id': 'req-233d183a-eff1-459b-b09f-6f868e9dcc61'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:48.027817 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359caec-06ec-493c-99b4-e3490ca9adaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:48.031434 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359caec-06ec-493c-99b4-e3490ca9adaf service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:48.031650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-233d183a-eff1-459b-b09f-6f868e9dcc61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3a70eb47fed4e959fc51400ddc3caf1', 'request_id': 'req-233d183a-eff1-459b-b09f-6f868e9dcc61'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:48.039327 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1063/2135] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:48.040684 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3d14b-7b4b-402c-88ef-99477a7e7757 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:48.040972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3d14b-7b4b-402c-88ef-99477a7e7757 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:48.041254 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3d14b-7b4b-402c-88ef-99477a7e7757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:48.043672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-233d183a-eff1-459b-b09f-6f868e9dcc61 admin admin] The token cache is being invalidate because project f3a70eb47fed4e959fc51400ddc3caf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:46:48.043960 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-233d183a-eff1-459b-b09f-6f868e9dcc61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:48.044705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-233d183a-eff1-459b-b09f-6f868e9dcc61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:48.046399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1073/2136] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:46:47 2022] DELETE /identity/v3/projects/f3a70eb47fed4e959fc51400ddc3caf1 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:48.051795 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d183a-eff1-459b-b09f-6f868e9dcc61 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:48.053162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-233d183a-eff1-459b-b09f-6f868e9dcc61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:48.062597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-69b3d14b-7b4b-402c-88ef-99477a7e7757 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:48.063776 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-69b3d14b-7b4b-402c-88ef-99477a7e7757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:48.125559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d183a-eff1-459b-b09f-6f868e9dcc61 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:48.130136 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1064/2137] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:48.130311 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e32e58-faa0-4f61-aaad-96064eec7740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:48.131165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e32e58-faa0-4f61-aaad-96064eec7740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:48.131484 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e32e58-faa0-4f61-aaad-96064eec7740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:48.131582 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028b460-fe3e-46bd-a90f-202b06ac655d None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:48.131819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028b460-fe3e-46bd-a90f-202b06ac655d None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:48.132116 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028b460-fe3e-46bd-a90f-202b06ac655d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:48.132445 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e32e58-faa0-4f61-aaad-96064eec7740 admin admin] PATH_INFO: `/v3/projects/ebc05582a9d547c5aca8136030e5f322` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:48.134267 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e32e58-faa0-4f61-aaad-96064eec7740 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebc05582a9d547c5aca8136030e5f322)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:48.136215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e32e58-faa0-4f61-aaad-96064eec7740 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:48.151041 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-8028b460-fe3e-46bd-a90f-202b06ac655d None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:48.152239 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-8028b460-fe3e-46bd-a90f-202b06ac655d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:48.168125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1065/2138] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:48.168125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8028b460-fe3e-46bd-a90f-202b06ac655d None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:48.177599 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8028b460-fe3e-46bd-a90f-202b06ac655d None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:48.183678 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b7740-fd73-43c1-824f-7c431ca0a926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:48.184323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b7740-fd73-43c1-824f-7c431ca0a926 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:48.184600 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b7740-fd73-43c1-824f-7c431ca0a926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:48.184874 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b7740-fd73-43c1-824f-7c431ca0a926 admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:48.185596 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b7740-fd73-43c1-824f-7c431ca0a926 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:46:48.185891 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b7740-fd73-43c1-824f-7c431ca0a926 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:48.187513 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b7740-fd73-43c1-824f-7c431ca0a926 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:48.199302 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1066/2139] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:46:48.205158 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8cdff-f0da-4227-8503-f69b4d285949 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:48.205158 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8cdff-f0da-4227-8503-f69b4d285949 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:48.205158 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8cdff-f0da-4227-8503-f69b4d285949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:48.218929 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-dcd8cdff-f0da-4227-8503-f69b4d285949 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:48.220111 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-dcd8cdff-f0da-4227-8503-f69b4d285949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:48.224933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:48.230732 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1067/2140] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:48.232592 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8cdff-f0da-4227-8503-f69b4d285949 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:48.240592 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8cdff-f0da-4227-8503-f69b4d285949 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:48.245957 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8dda0a-2809-4083-9c58-4acc2e118c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:48.246707 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dda0a-2809-4083-9c58-4acc2e118c41 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:48.247007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dda0a-2809-4083-9c58-4acc2e118c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:48.247267 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dda0a-2809-4083-9c58-4acc2e118c41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:48.247946 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8dda0a-2809-4083-9c58-4acc2e118c41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:48.249896 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8dda0a-2809-4083-9c58-4acc2e118c41 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:48.276508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-58e32e58-faa0-4f61-aaad-96064eec7740 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebc05582a9d547c5aca8136030e5f322', 'request_id': 'req-58e32e58-faa0-4f61-aaad-96064eec7740'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:48.429902 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1068/2141] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:46:48 2022] POST /identity/v3/projects => generated 383 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:48.429902 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8dda0a-2809-4083-9c58-4acc2e118c41 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:48.486240 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-58e32e58-faa0-4f61-aaad-96064eec7740 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebc05582a9d547c5aca8136030e5f322', 'request_id': 'req-58e32e58-faa0-4f61-aaad-96064eec7740'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:48.492779 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8dda0a-2809-4083-9c58-4acc2e118c41 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:48.493415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-58e32e58-faa0-4f61-aaad-96064eec7740 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebc05582a9d547c5aca8136030e5f322', 'request_id': 'req-58e32e58-faa0-4f61-aaad-96064eec7740'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:48.497771 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c0f55-8b60-4054-a956-52ad05d4e03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:48.498464 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c0f55-8b60-4054-a956-52ad05d4e03e admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:48.498754 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c0f55-8b60-4054-a956-52ad05d4e03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:48.499057 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c0f55-8b60-4054-a956-52ad05d4e03e admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:48.499780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c0f55-8b60-4054-a956-52ad05d4e03e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:46:48.500085 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c0f55-8b60-4054-a956-52ad05d4e03e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:48.501687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c0f55-8b60-4054-a956-52ad05d4e03e admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:48.505751 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-58e32e58-faa0-4f61-aaad-96064eec7740 admin admin] The token cache is being invalidate because project ebc05582a9d547c5aca8136030e5f322 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:46:48.506284 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-58e32e58-faa0-4f61-aaad-96064eec7740 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:48.507130 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-58e32e58-faa0-4f61-aaad-96064eec7740 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:48.508883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1074/2142] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:46:48 2022] DELETE /identity/v3/projects/ebc05582a9d547c5aca8136030e5f322 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:48.509776 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e32e58-faa0-4f61-aaad-96064eec7740 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:48.513879 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-58e32e58-faa0-4f61-aaad-96064eec7740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:48.517189 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1069/2143] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:46:48.518233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c0f55-8b60-4054-a956-52ad05d4e03e admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:48.520095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-ec8c0f55-8b60-4054-a956-52ad05d4e03e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:48.534046 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c0f55-8b60-4054-a956-52ad05d4e03e admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:48.540182 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6016e54-a08e-4f82-8a53-16d44a38114c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:48.541046 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6016e54-a08e-4f82-8a53-16d44a38114c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:48.541324 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6016e54-a08e-4f82-8a53-16d44a38114c admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:48.541597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6016e54-a08e-4f82-8a53-16d44a38114c admin admin] PATH_INFO: `/v3/users/0b005e3d155b4657b0a7d05d43081afc` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:48.542997 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6016e54-a08e-4f82-8a53-16d44a38114c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b005e3d155b4657b0a7d05d43081afc)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:48.544864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6016e54-a08e-4f82-8a53-16d44a38114c admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:48.582695 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e32e58-faa0-4f61-aaad-96064eec7740 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:48.587675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81868f1-098c-4818-957a-5d1492fa2cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:48.588316 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81868f1-098c-4818-957a-5d1492fa2cb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:48.588590 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81868f1-098c-4818-957a-5d1492fa2cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:48.588867 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81868f1-098c-4818-957a-5d1492fa2cb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:48.589555 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81868f1-098c-4818-957a-5d1492fa2cb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:48.591522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81868f1-098c-4818-957a-5d1492fa2cb4 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:48.719800 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1075/2144] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:46:48 2022] POST /identity/v3/users => generated 292 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:48.720544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81868f1-098c-4818-957a-5d1492fa2cb4 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:48.723857 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-f81868f1-098c-4818-957a-5d1492fa2cb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:48.739002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81868f1-098c-4818-957a-5d1492fa2cb4 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:48.745650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a876f-20b7-4837-a579-46f5461a7c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:48.746995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a876f-20b7-4837-a579-46f5461a7c94 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:48.747661 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a876f-20b7-4837-a579-46f5461a7c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:48.748119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a876f-20b7-4837-a579-46f5461a7c94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:48.750411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a876f-20b7-4837-a579-46f5461a7c94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:48.752363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a876f-20b7-4837-a579-46f5461a7c94 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:48.806831 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f6016e54-a08e-4f82-8a53-16d44a38114c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b005e3d155b4657b0a7d05d43081afc'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:48.813415 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f6016e54-a08e-4f82-8a53-16d44a38114c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b005e3d155b4657b0a7d05d43081afc'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:48.814067 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f6016e54-a08e-4f82-8a53-16d44a38114c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b005e3d155b4657b0a7d05d43081afc'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:48.828639 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f6016e54-a08e-4f82-8a53-16d44a38114c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b005e3d155b4657b0a7d05d43081afc'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:48.840596 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1076/2145] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:46:48 2022] POST /identity/v3/projects => generated 393 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:48.840596 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a876f-20b7-4837-a579-46f5461a7c94 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:48.903150 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a876f-20b7-4837-a579-46f5461a7c94 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:48.907928 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c95cb-9445-4cc2-ad40-8214297749d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:48.908582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c95cb-9445-4cc2-ad40-8214297749d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:48.908858 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c95cb-9445-4cc2-ad40-8214297749d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:48.909129 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c95cb-9445-4cc2-ad40-8214297749d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:48.909850 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c95cb-9445-4cc2-ad40-8214297749d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:48.911478 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c95cb-9445-4cc2-ad40-8214297749d1 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:48.921689 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1077/2146] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:48.922478 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c95cb-9445-4cc2-ad40-8214297749d1 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:48.931768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c95cb-9445-4cc2-ad40-8214297749d1 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:48.932132 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f6016e54-a08e-4f82-8a53-16d44a38114c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b005e3d155b4657b0a7d05d43081afc'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:48.936332 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d855a1-1996-40f4-a850-6c70151ef326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:48.940567 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d855a1-1996-40f4-a850-6c70151ef326 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:48.940567 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d855a1-1996-40f4-a850-6c70151ef326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:48.940567 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d855a1-1996-40f4-a850-6c70151ef326 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:48.940567 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d855a1-1996-40f4-a850-6c70151ef326 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:48.940884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1070/2147] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:46:48 2022] DELETE /identity/v3/users/0b005e3d155b4657b0a7d05d43081afc => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:48.940960 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d855a1-1996-40f4-a850-6c70151ef326 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:48.943511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6016e54-a08e-4f82-8a53-16d44a38114c admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:48.945223 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f6016e54-a08e-4f82-8a53-16d44a38114c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:49.014504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1078/2148] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:46:48 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:49.018430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d855a1-1996-40f4-a850-6c70151ef326 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:49.020491 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6016e54-a08e-4f82-8a53-16d44a38114c admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:49.021358 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-b8d855a1-1996-40f4-a850-6c70151ef326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:49.022947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:49.026257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c657d-f3fa-4050-97b5-628aff984294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:49.027295 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c657d-f3fa-4050-97b5-628aff984294 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:49.027424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c657d-f3fa-4050-97b5-628aff984294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:49.027728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c657d-f3fa-4050-97b5-628aff984294 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:49.028523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54c657d-f3fa-4050-97b5-628aff984294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:49.030311 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54c657d-f3fa-4050-97b5-628aff984294 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:49.036002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d855a1-1996-40f4-a850-6c70151ef326 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:49.041701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ec924-eb24-46d5-bded-0421e82b08a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:49.041876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1071/2149] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:49.042348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ec924-eb24-46d5-bded-0421e82b08a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:49.042727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ec924-eb24-46d5-bded-0421e82b08a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:49.043074 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ec924-eb24-46d5-bded-0421e82b08a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:49.043878 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ec924-eb24-46d5-bded-0421e82b08a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:49.045616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ec924-eb24-46d5-bded-0421e82b08a2 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:49.045764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c657d-f3fa-4050-97b5-628aff984294 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:49.056272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1079/2150] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:46:49.056548 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c657d-f3fa-4050-97b5-628aff984294 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:49.060163 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ec924-eb24-46d5-bded-0421e82b08a2 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:49.061638 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167440ff-a7b2-46f6-a12d-d98f9ba8009a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:49.062548 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167440ff-a7b2-46f6-a12d-d98f9ba8009a admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:49.062832 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167440ff-a7b2-46f6-a12d-d98f9ba8009a admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:49.063109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167440ff-a7b2-46f6-a12d-d98f9ba8009a admin admin] PATH_INFO: `/v3/projects/e0d29951848d48489e69c93ed8ff42f5/users/26ea55c08d1b46228e85ed0b7ff733f7/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:49.070758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ec924-eb24-46d5-bded-0421e82b08a2 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:49.075471 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482deedc-58d0-4f01-a4b5-6b1758049009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:49.076120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482deedc-58d0-4f01-a4b5-6b1758049009 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:49.076441 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482deedc-58d0-4f01-a4b5-6b1758049009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:49.076737 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482deedc-58d0-4f01-a4b5-6b1758049009 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:49.077517 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482deedc-58d0-4f01-a4b5-6b1758049009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:49.079231 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482deedc-58d0-4f01-a4b5-6b1758049009 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:49.082102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167440ff-a7b2-46f6-a12d-d98f9ba8009a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0d29951848d48489e69c93ed8ff42f5, user_id=26ea55c08d1b46228e85ed0b7ff733f7, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:49.082830 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167440ff-a7b2-46f6-a12d-d98f9ba8009a admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:49.092398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1080/2151] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:49.096809 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482deedc-58d0-4f01-a4b5-6b1758049009 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:49.107819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482deedc-58d0-4f01-a4b5-6b1758049009 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:49.112589 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53256c0-3205-4939-aa49-fa63f2cec9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:49.114732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53256c0-3205-4939-aa49-fa63f2cec9fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:49.114732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53256c0-3205-4939-aa49-fa63f2cec9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:49.114732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53256c0-3205-4939-aa49-fa63f2cec9fd admin admin] PATH_INFO: `/v3/projects/e03ab1e3fbe7445da087c631d8af557b/users/78b6599200a94a2a8bcc0149dc22944d/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:49.128936 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53256c0-3205-4939-aa49-fa63f2cec9fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e03ab1e3fbe7445da087c631d8af557b, user_id=78b6599200a94a2a8bcc0149dc22944d, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:49.131674 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53256c0-3205-4939-aa49-fa63f2cec9fd admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:49.180134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1072/2152] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:49 2022] PUT /identity/v3/projects/e0d29951848d48489e69c93ed8ff42f5/users/26ea55c08d1b46228e85ed0b7ff733f7/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:46:49.192610 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616170ea-869a-441e-85f7-a4dec88d2100 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:49.194019 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616170ea-869a-441e-85f7-a4dec88d2100 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:49.194019 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616170ea-869a-441e-85f7-a4dec88d2100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:49.223010 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-616170ea-869a-441e-85f7-a4dec88d2100 None None] MFA Rules not processed for user `26ea55c08d1b46228e85ed0b7ff733f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:49.224533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-616170ea-869a-441e-85f7-a4dec88d2100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:49.271327 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1081/2153] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:49 2022] PUT /identity/v3/projects/e03ab1e3fbe7445da087c631d8af557b/users/78b6599200a94a2a8bcc0149dc22944d/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:46:49.275355 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40e360-b71d-4f41-a5a3-56f6bf047c00 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:49.275949 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40e360-b71d-4f41-a5a3-56f6bf047c00 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:49.276392 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40e360-b71d-4f41-a5a3-56f6bf047c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:49.297269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-da40e360-b71d-4f41-a5a3-56f6bf047c00 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:49.299149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-da40e360-b71d-4f41-a5a3-56f6bf047c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:49.309454 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1073/2154] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:46:49.317195 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616170ea-869a-441e-85f7-a4dec88d2100 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:49.320468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:49.381240 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1082/2155] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:46:49.389834 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40e360-b71d-4f41-a5a3-56f6bf047c00 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:49.391364 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-da40e360-b71d-4f41-a5a3-56f6bf047c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:49.406070 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616170ea-869a-441e-85f7-a4dec88d2100 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:49.411415 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1ba6e-1d7b-4a83-9ca9-93a396ef359e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:49.412064 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1ba6e-1d7b-4a83-9ca9-93a396ef359e admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:49.412353 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1ba6e-1d7b-4a83-9ca9-93a396ef359e admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:49.412783 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1ba6e-1d7b-4a83-9ca9-93a396ef359e admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:49.413510 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1ba6e-1d7b-4a83-9ca9-93a396ef359e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:49.415603 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1ba6e-1d7b-4a83-9ca9-93a396ef359e admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:49.459141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40e360-b71d-4f41-a5a3-56f6bf047c00 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:49.467991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10371f-1138-46f4-adb0-e31fa7cf294a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:49.467991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10371f-1138-46f4-adb0-e31fa7cf294a service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:49.467991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10371f-1138-46f4-adb0-e31fa7cf294a service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:49.467991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10371f-1138-46f4-adb0-e31fa7cf294a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:49.478206 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10371f-1138-46f4-adb0-e31fa7cf294a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:49.480230 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10371f-1138-46f4-adb0-e31fa7cf294a service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:49.495600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1083/2156] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:46:49.496628 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10371f-1138-46f4-adb0-e31fa7cf294a service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:49.498476 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ac10371f-1138-46f4-adb0-e31fa7cf294a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:49.513036 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10371f-1138-46f4-adb0-e31fa7cf294a service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:49.519668 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae839f48-ef6f-4952-a28a-48ff584b0a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:49.520525 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae839f48-ef6f-4952-a28a-48ff584b0a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:49.520805 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae839f48-ef6f-4952-a28a-48ff584b0a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:49.521078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae839f48-ef6f-4952-a28a-48ff584b0a83 admin admin] PATH_INFO: `/v3/projects/d55a30e38f534f679e221b42ffec616c` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:49.524161 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae839f48-ef6f-4952-a28a-48ff584b0a83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d55a30e38f534f679e221b42ffec616c)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:49.529308 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae839f48-ef6f-4952-a28a-48ff584b0a83 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:49.538371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1074/2157] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:46:49 2022] POST /identity/v3/projects => generated 381 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:46:49.539466 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1ba6e-1d7b-4a83-9ca9-93a396ef359e admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:49.613244 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1ba6e-1d7b-4a83-9ca9-93a396ef359e admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:49.621636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fd35c-8f0c-40c6-8e4c-802269318b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:49.621636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fd35c-8f0c-40c6-8e4c-802269318b3d admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:49.621636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fd35c-8f0c-40c6-8e4c-802269318b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:49.621636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fd35c-8f0c-40c6-8e4c-802269318b3d admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:49.621989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394fd35c-8f0c-40c6-8e4c-802269318b3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:49.622321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394fd35c-8f0c-40c6-8e4c-802269318b3d admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:49.661940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-ae839f48-ef6f-4952-a28a-48ff584b0a83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd55a30e38f534f679e221b42ffec616c', 'request_id': 'req-ae839f48-ef6f-4952-a28a-48ff584b0a83'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:49.737103 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1075/2158] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:46:49 2022] POST /identity/v3/users => generated 297 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:49.743104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fd35c-8f0c-40c6-8e4c-802269318b3d admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:49.755245 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-ae839f48-ef6f-4952-a28a-48ff584b0a83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd55a30e38f534f679e221b42ffec616c', 'request_id': 'req-ae839f48-ef6f-4952-a28a-48ff584b0a83'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:49.759778 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fd35c-8f0c-40c6-8e4c-802269318b3d admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:49.764633 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-ae839f48-ef6f-4952-a28a-48ff584b0a83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd55a30e38f534f679e221b42ffec616c', 'request_id': 'req-ae839f48-ef6f-4952-a28a-48ff584b0a83'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:49.774740 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-ae839f48-ef6f-4952-a28a-48ff584b0a83 admin admin] The token cache is being invalidate because project d55a30e38f534f679e221b42ffec616c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:46:49.774958 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-ae839f48-ef6f-4952-a28a-48ff584b0a83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:49.775622 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-ae839f48-ef6f-4952-a28a-48ff584b0a83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:49.777534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1084/2159] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:46:49 2022] DELETE /identity/v3/projects/d55a30e38f534f679e221b42ffec616c => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:49.782453 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f793d3-2d53-416b-80dc-ae75da6dd3db None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:49.782690 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f793d3-2d53-416b-80dc-ae75da6dd3db None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:49.782975 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f793d3-2d53-416b-80dc-ae75da6dd3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:49.806622 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-83f793d3-2d53-416b-80dc-ae75da6dd3db None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:49.808209 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-83f793d3-2d53-416b-80dc-ae75da6dd3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:49.839115 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb9e13-aff6-4d1a-9e2b-39fa0dd7f2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:49.839834 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb9e13-aff6-4d1a-9e2b-39fa0dd7f2c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:49.840068 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb9e13-aff6-4d1a-9e2b-39fa0dd7f2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:49.840396 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb9e13-aff6-4d1a-9e2b-39fa0dd7f2c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:49.841137 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb9e13-aff6-4d1a-9e2b-39fa0dd7f2c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:49.842803 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb9e13-aff6-4d1a-9e2b-39fa0dd7f2c6 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:49.864214 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1076/2160] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:46:49.864870 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb9e13-aff6-4d1a-9e2b-39fa0dd7f2c6 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:49.868363 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-57bb9e13-aff6-4d1a-9e2b-39fa0dd7f2c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:49.900914 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb9e13-aff6-4d1a-9e2b-39fa0dd7f2c6 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:49.907227 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1085/2161] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:46:49.915677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb9dc6-fa95-49c3-8e6b-fcf40ab4af7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:49.916759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb9dc6-fa95-49c3-8e6b-fcf40ab4af7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:49.916897 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5822f-c197-406f-beac-ef2880b30741 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:49.917044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb9dc6-fa95-49c3-8e6b-fcf40ab4af7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:49.917165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5822f-c197-406f-beac-ef2880b30741 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:49.917318 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb9dc6-fa95-49c3-8e6b-fcf40ab4af7c admin admin] PATH_INFO: `/v3/projects/5613d8ab1a214fbb9b173eaf13af5d33/users/c5255915950d4d629bf9633022014e51/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:49.917431 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5822f-c197-406f-beac-ef2880b30741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:49.942257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-7be5822f-c197-406f-beac-ef2880b30741 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:49.943600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-7be5822f-c197-406f-beac-ef2880b30741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:49.954065 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1086/2162] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:49.966501 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb9dc6-fa95-49c3-8e6b-fcf40ab4af7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5613d8ab1a214fbb9b173eaf13af5d33, user_id=c5255915950d4d629bf9633022014e51, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:49.969209 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5822f-c197-406f-beac-ef2880b30741 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:49.969602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb9dc6-fa95-49c3-8e6b-fcf40ab4af7c admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:49.978371 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5822f-c197-406f-beac-ef2880b30741 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:49.983661 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea054373-a105-49a0-9254-9ad5b9122c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:49.984369 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea054373-a105-49a0-9254-9ad5b9122c3f admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:49.988001 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea054373-a105-49a0-9254-9ad5b9122c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:49.988001 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea054373-a105-49a0-9254-9ad5b9122c3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:49.988001 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea054373-a105-49a0-9254-9ad5b9122c3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:46:49.988001 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea054373-a105-49a0-9254-9ad5b9122c3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:49.988001 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea054373-a105-49a0-9254-9ad5b9122c3f admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:50.000470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1087/2163] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:46:50.004868 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea054373-a105-49a0-9254-9ad5b9122c3f admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:50.016790 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea054373-a105-49a0-9254-9ad5b9122c3f admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:50.027465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a48e1-44a6-443e-8f54-f13e757b4376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:50.028148 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a48e1-44a6-443e-8f54-f13e757b4376 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:50.028433 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a48e1-44a6-443e-8f54-f13e757b4376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:50.028735 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a48e1-44a6-443e-8f54-f13e757b4376 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:50.029430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a48e1-44a6-443e-8f54-f13e757b4376 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:50.031423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a48e1-44a6-443e-8f54-f13e757b4376 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:50.108487 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1077/2164] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:49 2022] PUT /identity/v3/projects/5613d8ab1a214fbb9b173eaf13af5d33/users/c5255915950d4d629bf9633022014e51/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:50.112424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb9dc6-fa95-49c3-8e6b-fcf40ab4af7c admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:50.169974 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1088/2165] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:46:50 2022] POST /identity/v3/projects => generated 365 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:46:50.173975 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a48e1-44a6-443e-8f54-f13e757b4376 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:50.203152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb9dc6-fa95-49c3-8e6b-fcf40ab4af7c admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:50.203946 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:50.270989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a622ae-2b48-48a3-9c4c-63cda96b3c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:50.272034 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a622ae-2b48-48a3-9c4c-63cda96b3c0a admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:50.273042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a622ae-2b48-48a3-9c4c-63cda96b3c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:50.273391 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a622ae-2b48-48a3-9c4c-63cda96b3c0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:50.274205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a622ae-2b48-48a3-9c4c-63cda96b3c0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:50.275997 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a622ae-2b48-48a3-9c4c-63cda96b3c0a admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:50.288503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1078/2166] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:46:50.291374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a622ae-2b48-48a3-9c4c-63cda96b3c0a admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:50.295014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a48e1-44a6-443e-8f54-f13e757b4376 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:50.299920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2eca15-3992-4478-a506-d98dea703da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:50.300612 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2eca15-3992-4478-a506-d98dea703da2 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:50.300899 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2eca15-3992-4478-a506-d98dea703da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:50.301178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2eca15-3992-4478-a506-d98dea703da2 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:50.301303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a622ae-2b48-48a3-9c4c-63cda96b3c0a admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:50.302162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2eca15-3992-4478-a506-d98dea703da2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:50.304836 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2eca15-3992-4478-a506-d98dea703da2 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:50.306763 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c27e55-5b1e-4022-bba7-94a5eb8c547d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:50.307696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c27e55-5b1e-4022-bba7-94a5eb8c547d admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:50.307989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c27e55-5b1e-4022-bba7-94a5eb8c547d admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:50.308265 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c27e55-5b1e-4022-bba7-94a5eb8c547d admin admin] PATH_INFO: `/v3/projects/5613d8ab1a214fbb9b173eaf13af5d33/users/c5255915950d4d629bf9633022014e51/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:50.310312 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c27e55-5b1e-4022-bba7-94a5eb8c547d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5613d8ab1a214fbb9b173eaf13af5d33, user_id=c5255915950d4d629bf9633022014e51, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:50.312978 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c27e55-5b1e-4022-bba7-94a5eb8c547d admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:50.329604 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1079/2167] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:50 2022] PUT /identity/v3/projects/5613d8ab1a214fbb9b173eaf13af5d33/users/c5255915950d4d629bf9633022014e51/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:46:50.341125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd5d14-2c49-48f6-acbe-d6c29c4d4d82 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:50.341410 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd5d14-2c49-48f6-acbe-d6c29c4d4d82 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:50.341692 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd5d14-2c49-48f6-acbe-d6c29c4d4d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:50.371605 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-c1dd5d14-2c49-48f6-acbe-d6c29c4d4d82 None None] MFA Rules not processed for user `c5255915950d4d629bf9633022014e51`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:50.372922 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-c1dd5d14-2c49-48f6-acbe-d6c29c4d4d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:50.380797 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1089/2168] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:46:50 2022] POST /identity/v3/users => generated 283 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:46:50.384627 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2eca15-3992-4478-a506-d98dea703da2 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:50.395713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2eca15-3992-4478-a506-d98dea703da2 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:50.406589 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:50.458164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1080/2169] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:46:50.467085 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29735c7-dd6c-4dbf-a5c7-6c5dd5923ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:50.467861 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29735c7-dd6c-4dbf-a5c7-6c5dd5923ecb admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:50.468240 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29735c7-dd6c-4dbf-a5c7-6c5dd5923ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:50.468637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29735c7-dd6c-4dbf-a5c7-6c5dd5923ecb admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:50.469514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29735c7-dd6c-4dbf-a5c7-6c5dd5923ecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:50.471228 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29735c7-dd6c-4dbf-a5c7-6c5dd5923ecb admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:50.476765 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd5d14-2c49-48f6-acbe-d6c29c4d4d82 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:50.479015 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-c1dd5d14-2c49-48f6-acbe-d6c29c4d4d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:50.485902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1090/2170] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:46:50.489663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29735c7-dd6c-4dbf-a5c7-6c5dd5923ecb admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:50.504094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29735c7-dd6c-4dbf-a5c7-6c5dd5923ecb admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:50.510169 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae20b3-0de6-487c-b698-366872619b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:50.510936 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae20b3-0de6-487c-b698-366872619b31 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:50.511356 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae20b3-0de6-487c-b698-366872619b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:50.512372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae20b3-0de6-487c-b698-366872619b31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:50.512372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae20b3-0de6-487c-b698-366872619b31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:50.514177 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae20b3-0de6-487c-b698-366872619b31 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:50.530059 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1091/2171] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:46:50.531149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae20b3-0de6-487c-b698-366872619b31 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:50.541605 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae20b3-0de6-487c-b698-366872619b31 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:50.544123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd5d14-2c49-48f6-acbe-d6c29c4d4d82 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:50.546370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ad840-8152-4943-9126-b684b42a08e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:50.547260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ad840-8152-4943-9126-b684b42a08e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:50.547542 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ad840-8152-4943-9126-b684b42a08e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:50.547816 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ad840-8152-4943-9126-b684b42a08e1 admin admin] PATH_INFO: `/v3/projects/46727708a5de48f08e1e6b9cc4b7a82b/users/618e79413e50472c80729cf2ac970109/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:50.548014 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b16455-8d34-4b57-b34c-048e0e9bb143 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:50.548476 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b16455-8d34-4b57-b34c-048e0e9bb143 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:50.548761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b16455-8d34-4b57-b34c-048e0e9bb143 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:50.549040 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b16455-8d34-4b57-b34c-048e0e9bb143 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:50.559256 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b16455-8d34-4b57-b34c-048e0e9bb143 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:50.560741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b16455-8d34-4b57-b34c-048e0e9bb143 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:50.563688 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ad840-8152-4943-9126-b684b42a08e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46727708a5de48f08e1e6b9cc4b7a82b, user_id=618e79413e50472c80729cf2ac970109, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:50.566468 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ad840-8152-4943-9126-b684b42a08e1 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:50.571628 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1081/2172] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:50.685452 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1092/2173] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:50 2022] PUT /identity/v3/projects/46727708a5de48f08e1e6b9cc4b7a82b/users/618e79413e50472c80729cf2ac970109/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:50.698866 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2638d3-3b0b-4cbf-9a4e-e18a7af7756e None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:50.699205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2638d3-3b0b-4cbf-9a4e-e18a7af7756e None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:50.699513 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2638d3-3b0b-4cbf-9a4e-e18a7af7756e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:50.729422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-ba2638d3-3b0b-4cbf-9a4e-e18a7af7756e None None] MFA Rules not processed for user `618e79413e50472c80729cf2ac970109`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:50.730147 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ad840-8152-4943-9126-b684b42a08e1 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:50.731157 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-ba2638d3-3b0b-4cbf-9a4e-e18a7af7756e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:50.732521 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-702ad840-8152-4943-9126-b684b42a08e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:50.794940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ad840-8152-4943-9126-b684b42a08e1 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:50.798885 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e22a8c-2bbd-484c-8e56-3c3bd5d36b79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:50.799622 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e22a8c-2bbd-484c-8e56-3c3bd5d36b79 service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:50.799967 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e22a8c-2bbd-484c-8e56-3c3bd5d36b79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:50.800301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e22a8c-2bbd-484c-8e56-3c3bd5d36b79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:50.802220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-44e22a8c-2bbd-484c-8e56-3c3bd5d36b79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:50.810258 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1082/2174] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:50 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:50.822324 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2638d3-3b0b-4cbf-9a4e-e18a7af7756e None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:50.832028 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2638d3-3b0b-4cbf-9a4e-e18a7af7756e None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:50.835932 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595ddfd-ccf9-4cec-bf29-bd78ae8ccb27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:50.836617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595ddfd-ccf9-4cec-bf29-bd78ae8ccb27 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:50.836901 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595ddfd-ccf9-4cec-bf29-bd78ae8ccb27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:50.837176 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595ddfd-ccf9-4cec-bf29-bd78ae8ccb27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:50.845642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595ddfd-ccf9-4cec-bf29-bd78ae8ccb27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:50.847095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595ddfd-ccf9-4cec-bf29-bd78ae8ccb27 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:50.857783 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1083/2175] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:50 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:46:50.872748 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e22a8c-2bbd-484c-8e56-3c3bd5d36b79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:50.874339 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e22a8c-2bbd-484c-8e56-3c3bd5d36b79 service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:50.885735 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1093/2176] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:51.455166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595ddfd-ccf9-4cec-bf29-bd78ae8ccb27 service cinder] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:51.457302 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-6595ddfd-ccf9-4cec-bf29-bd78ae8ccb27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:51.481273 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e22a8c-2bbd-484c-8e56-3c3bd5d36b79 service cinder] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:51.549744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595ddfd-ccf9-4cec-bf29-bd78ae8ccb27 service cinder] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:51.562032 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e22a8c-2bbd-484c-8e56-3c3bd5d36b79 service cinder] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:51.566875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bccc5b-72ad-428b-a7ba-942f804481ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:51.567729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bccc5b-72ad-428b-a7ba-942f804481ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:51.567987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bccc5b-72ad-428b-a7ba-942f804481ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:51.568239 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bccc5b-72ad-428b-a7ba-942f804481ea admin admin] PATH_INFO: `/v3/users/26ea55c08d1b46228e85ed0b7ff733f7` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:51.569999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bccc5b-72ad-428b-a7ba-942f804481ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=26ea55c08d1b46228e85ed0b7ff733f7)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:51.571727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aac517-b0f8-47fa-9a62-e75e470637ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:51.572114 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bccc5b-72ad-428b-a7ba-942f804481ea admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:51.574596 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aac517-b0f8-47fa-9a62-e75e470637ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:51.575727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aac517-b0f8-47fa-9a62-e75e470637ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:51.576075 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aac517-b0f8-47fa-9a62-e75e470637ae admin admin] PATH_INFO: `/v3/users/8ae868193ad64b638e2e2a1cc1371980` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:51.578500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aac517-b0f8-47fa-9a62-e75e470637ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ae868193ad64b638e2e2a1cc1371980)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:51.582602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aac517-b0f8-47fa-9a62-e75e470637ae admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:51.648775 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:51.756822 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:51.836932 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-45bccc5b-72ad-428b-a7ba-942f804481ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26ea55c08d1b46228e85ed0b7ff733f7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:51.854557 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-45bccc5b-72ad-428b-a7ba-942f804481ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26ea55c08d1b46228e85ed0b7ff733f7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:51.855032 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-45bccc5b-72ad-428b-a7ba-942f804481ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26ea55c08d1b46228e85ed0b7ff733f7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:51.856000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-45bccc5b-72ad-428b-a7ba-942f804481ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26ea55c08d1b46228e85ed0b7ff733f7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:51.870037 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-51aac517-b0f8-47fa-9a62-e75e470637ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ae868193ad64b638e2e2a1cc1371980'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:51.883310 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-45bccc5b-72ad-428b-a7ba-942f804481ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26ea55c08d1b46228e85ed0b7ff733f7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:51.887289 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-51aac517-b0f8-47fa-9a62-e75e470637ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ae868193ad64b638e2e2a1cc1371980'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:51.889522 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-51aac517-b0f8-47fa-9a62-e75e470637ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ae868193ad64b638e2e2a1cc1371980'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:51.910672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-51aac517-b0f8-47fa-9a62-e75e470637ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ae868193ad64b638e2e2a1cc1371980'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:52.013100 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1094/2177] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:46:51 2022] DELETE /identity/v3/users/26ea55c08d1b46228e85ed0b7ff733f7 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:52.028268 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80597ace-5d3e-4d1c-9070-e3f040c32118 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:52.028572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80597ace-5d3e-4d1c-9070-e3f040c32118 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:52.028906 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80597ace-5d3e-4d1c-9070-e3f040c32118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:52.050926 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-80597ace-5d3e-4d1c-9070-e3f040c32118 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:52.052949 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-80597ace-5d3e-4d1c-9070-e3f040c32118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:52.120882 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-51aac517-b0f8-47fa-9a62-e75e470637ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ae868193ad64b638e2e2a1cc1371980'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:52.124389 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1095/2178] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:52.130000 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1084/2179] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:46:51 2022] DELETE /identity/v3/users/8ae868193ad64b638e2e2a1cc1371980 => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:52.136712 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aac517-b0f8-47fa-9a62-e75e470637ae admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:52.137468 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a2fac-9d16-459d-a45b-62605bc402d3 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:52.138042 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a2fac-9d16-459d-a45b-62605bc402d3 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:52.138184 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-51aac517-b0f8-47fa-9a62-e75e470637ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:52.138328 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a2fac-9d16-459d-a45b-62605bc402d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:52.164904 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-c96a2fac-9d16-459d-a45b-62605bc402d3 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:52.166107 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c96a2fac-9d16-459d-a45b-62605bc402d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:52.232417 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aac517-b0f8-47fa-9a62-e75e470637ae admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:52.236492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c41dbc-2e73-4581-995a-fe2975b0ab08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:52.237308 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1096/2180] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:52.237434 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c41dbc-2e73-4581-995a-fe2975b0ab08 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:52.237513 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c41dbc-2e73-4581-995a-fe2975b0ab08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:52.237865 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c41dbc-2e73-4581-995a-fe2975b0ab08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:52.240443 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-02c41dbc-2e73-4581-995a-fe2975b0ab08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:52.248449 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a2fac-9d16-459d-a45b-62605bc402d3 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:52.250054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c96a2fac-9d16-459d-a45b-62605bc402d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:52.255192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c41dbc-2e73-4581-995a-fe2975b0ab08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:52.256813 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c41dbc-2e73-4581-995a-fe2975b0ab08 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:52.263956 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a2fac-9d16-459d-a45b-62605bc402d3 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:52.267718 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1085/2181] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:52.269047 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5464c9-a7f6-46ed-a407-906ea1695a92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:52.269789 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5464c9-a7f6-46ed-a407-906ea1695a92 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:52.270279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5464c9-a7f6-46ed-a407-906ea1695a92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:52.270594 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5464c9-a7f6-46ed-a407-906ea1695a92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:52.283581 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5464c9-a7f6-46ed-a407-906ea1695a92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:52.285250 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5464c9-a7f6-46ed-a407-906ea1695a92 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:52.296655 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1097/2182] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:52.446790 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:53.002708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c41dbc-2e73-4581-995a-fe2975b0ab08 service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:53.004554 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-02c41dbc-2e73-4581-995a-fe2975b0ab08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:53.023205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c41dbc-2e73-4581-995a-fe2975b0ab08 service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:53.040678 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c8b61-2366-4cf8-b919-0cf22a2f68be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:53.042880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c8b61-2366-4cf8-b919-0cf22a2f68be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:53.043383 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c8b61-2366-4cf8-b919-0cf22a2f68be admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:53.043837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c8b61-2366-4cf8-b919-0cf22a2f68be admin admin] PATH_INFO: `/v3/projects/e0d29951848d48489e69c93ed8ff42f5` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:53.045359 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967c8b61-2366-4cf8-b919-0cf22a2f68be admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0d29951848d48489e69c93ed8ff42f5)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:53.047482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967c8b61-2366-4cf8-b919-0cf22a2f68be admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:53.078274 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5464c9-a7f6-46ed-a407-906ea1695a92 service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:53.080480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-3e5464c9-a7f6-46ed-a407-906ea1695a92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:53.099485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5464c9-a7f6-46ed-a407-906ea1695a92 service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:53.105529 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2088a-d4b3-4480-b4fd-fd177c56d1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:53.106816 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2088a-d4b3-4480-b4fd-fd177c56d1d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:53.107102 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2088a-d4b3-4480-b4fd-fd177c56d1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:53.107376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2088a-d4b3-4480-b4fd-fd177c56d1d3 admin admin] PATH_INFO: `/v3/projects/e478db03382f437fb83d6b1e74441419` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:53.109111 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2088a-d4b3-4480-b4fd-fd177c56d1d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e478db03382f437fb83d6b1e74441419)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:53.112803 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2088a-d4b3-4480-b4fd-fd177c56d1d3 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:53.202095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-967c8b61-2366-4cf8-b919-0cf22a2f68be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0d29951848d48489e69c93ed8ff42f5', 'request_id': 'req-967c8b61-2366-4cf8-b919-0cf22a2f68be'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:53.279197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-967c8b61-2366-4cf8-b919-0cf22a2f68be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0d29951848d48489e69c93ed8ff42f5', 'request_id': 'req-967c8b61-2366-4cf8-b919-0cf22a2f68be'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:53.280374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-12e2088a-d4b3-4480-b4fd-fd177c56d1d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e478db03382f437fb83d6b1e74441419', 'request_id': 'req-12e2088a-d4b3-4480-b4fd-fd177c56d1d3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:53.294493 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-967c8b61-2366-4cf8-b919-0cf22a2f68be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0d29951848d48489e69c93ed8ff42f5', 'request_id': 'req-967c8b61-2366-4cf8-b919-0cf22a2f68be'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:53.314139 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-967c8b61-2366-4cf8-b919-0cf22a2f68be admin admin] The token cache is being invalidate because project e0d29951848d48489e69c93ed8ff42f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:46:53.314450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-967c8b61-2366-4cf8-b919-0cf22a2f68be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:53.316564 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-967c8b61-2366-4cf8-b919-0cf22a2f68be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:53.319628 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1086/2183] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:46:53 2022] DELETE /identity/v3/projects/e0d29951848d48489e69c93ed8ff42f5 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:53.323452 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c8b61-2366-4cf8-b919-0cf22a2f68be admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:53.326544 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-967c8b61-2366-4cf8-b919-0cf22a2f68be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:53.387201 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-12e2088a-d4b3-4480-b4fd-fd177c56d1d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e478db03382f437fb83d6b1e74441419', 'request_id': 'req-12e2088a-d4b3-4480-b4fd-fd177c56d1d3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:53.393133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-12e2088a-d4b3-4480-b4fd-fd177c56d1d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e478db03382f437fb83d6b1e74441419', 'request_id': 'req-12e2088a-d4b3-4480-b4fd-fd177c56d1d3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:53.411388 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-12e2088a-d4b3-4480-b4fd-fd177c56d1d3 admin admin] The token cache is being invalidate because project e478db03382f437fb83d6b1e74441419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:46:53.411772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-12e2088a-d4b3-4480-b4fd-fd177c56d1d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:53.413212 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-12e2088a-d4b3-4480-b4fd-fd177c56d1d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:53.415339 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1098/2184] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:46:53 2022] DELETE /identity/v3/projects/e478db03382f437fb83d6b1e74441419 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:46:53.432381 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c8b61-2366-4cf8-b919-0cf22a2f68be admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:53.518107 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d50157-461b-4726-acac-369b963d7859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:53.519227 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d50157-461b-4726-acac-369b963d7859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:53.519513 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d50157-461b-4726-acac-369b963d7859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:53.520172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d50157-461b-4726-acac-369b963d7859 admin admin] PATH_INFO: `/v3/users/c5255915950d4d629bf9633022014e51` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:53.521984 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d50157-461b-4726-acac-369b963d7859 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5255915950d4d629bf9633022014e51)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:53.525188 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:53.534715 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d50157-461b-4726-acac-369b963d7859 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:53.777266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-46d50157-461b-4726-acac-369b963d7859 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5255915950d4d629bf9633022014e51'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:53.792318 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-46d50157-461b-4726-acac-369b963d7859 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5255915950d4d629bf9633022014e51'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:53.794628 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-46d50157-461b-4726-acac-369b963d7859 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5255915950d4d629bf9633022014e51'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:53.820295 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-46d50157-461b-4726-acac-369b963d7859 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5255915950d4d629bf9633022014e51'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:53.894752 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-46d50157-461b-4726-acac-369b963d7859 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5255915950d4d629bf9633022014e51'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:53.905973 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1087/2185] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:46:53 2022] DELETE /identity/v3/users/c5255915950d4d629bf9633022014e51 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:54.258882 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:54.520555 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2088a-d4b3-4480-b4fd-fd177c56d1d3 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:54.522587 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-12e2088a-d4b3-4480-b4fd-fd177c56d1d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:54.558111 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2088a-d4b3-4480-b4fd-fd177c56d1d3 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:54.626628 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657b99c-e6f7-4d8d-bac8-c0fd3a194443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:54.627499 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657b99c-e6f7-4d8d-bac8-c0fd3a194443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:54.627783 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657b99c-e6f7-4d8d-bac8-c0fd3a194443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:54.628060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657b99c-e6f7-4d8d-bac8-c0fd3a194443 admin admin] PATH_INFO: `/v3/projects/5613d8ab1a214fbb9b173eaf13af5d33` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:54.629268 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657b99c-e6f7-4d8d-bac8-c0fd3a194443 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5613d8ab1a214fbb9b173eaf13af5d33)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:54.631791 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657b99c-e6f7-4d8d-bac8-c0fd3a194443 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:54.751437 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-d657b99c-e6f7-4d8d-bac8-c0fd3a194443 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5613d8ab1a214fbb9b173eaf13af5d33', 'request_id': 'req-d657b99c-e6f7-4d8d-bac8-c0fd3a194443'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:54.809289 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-d657b99c-e6f7-4d8d-bac8-c0fd3a194443 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5613d8ab1a214fbb9b173eaf13af5d33', 'request_id': 'req-d657b99c-e6f7-4d8d-bac8-c0fd3a194443'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:54.819710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-d657b99c-e6f7-4d8d-bac8-c0fd3a194443 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5613d8ab1a214fbb9b173eaf13af5d33', 'request_id': 'req-d657b99c-e6f7-4d8d-bac8-c0fd3a194443'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:54.843518 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-d657b99c-e6f7-4d8d-bac8-c0fd3a194443 admin admin] The token cache is being invalidate because project 5613d8ab1a214fbb9b173eaf13af5d33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:46:54.843855 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-d657b99c-e6f7-4d8d-bac8-c0fd3a194443 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:54.845820 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-d657b99c-e6f7-4d8d-bac8-c0fd3a194443 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:46:54.847938 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1099/2186] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:46:54 2022] DELETE /identity/v3/projects/5613d8ab1a214fbb9b173eaf13af5d33 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:54.854414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78271e-477d-49ac-91b4-b2efc816bd92 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:54.854414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78271e-477d-49ac-91b4-b2efc816bd92 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:54.855101 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78271e-477d-49ac-91b4-b2efc816bd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:54.877714 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-8d78271e-477d-49ac-91b4-b2efc816bd92 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:54.878996 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-8d78271e-477d-49ac-91b4-b2efc816bd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:54.991964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1088/2187] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:54.999975 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024661b-c1dd-4912-b854-e77270bda6df None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:55.000268 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024661b-c1dd-4912-b854-e77270bda6df None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:55.000546 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2024661b-c1dd-4912-b854-e77270bda6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:55.028757 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-2024661b-c1dd-4912-b854-e77270bda6df None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:55.030125 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-2024661b-c1dd-4912-b854-e77270bda6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:55.040225 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1100/2188] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:55.044755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78271e-477d-49ac-91b4-b2efc816bd92 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:55.056119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78271e-477d-49ac-91b4-b2efc816bd92 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:55.061347 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45026de8-b2b8-429a-8819-14d881e52d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:55.062016 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45026de8-b2b8-429a-8819-14d881e52d55 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:55.062365 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45026de8-b2b8-429a-8819-14d881e52d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:55.062696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45026de8-b2b8-429a-8819-14d881e52d55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:55.063461 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45026de8-b2b8-429a-8819-14d881e52d55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:46:55.063759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45026de8-b2b8-429a-8819-14d881e52d55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:55.066077 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45026de8-b2b8-429a-8819-14d881e52d55 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:55.088945 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1089/2189] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:46:55.094011 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024661b-c1dd-4912-b854-e77270bda6df None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:55.104453 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2024661b-c1dd-4912-b854-e77270bda6df None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:55.110817 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193c97f-070a-4050-995a-96f430aee33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:55.111480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193c97f-070a-4050-995a-96f430aee33f admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:55.112061 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193c97f-070a-4050-995a-96f430aee33f admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:55.112480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193c97f-070a-4050-995a-96f430aee33f admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:55.113479 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193c97f-070a-4050-995a-96f430aee33f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:55.116153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193c97f-070a-4050-995a-96f430aee33f admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:55.226227 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1101/2190] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:46:55 2022] POST /identity/v3/projects => generated 381 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:55.229812 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45026de8-b2b8-429a-8819-14d881e52d55 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:55.242124 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45026de8-b2b8-429a-8819-14d881e52d55 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:55.319433 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54156051-1272-461f-a7d5-c245d2257500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:55.324910 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54156051-1272-461f-a7d5-c245d2257500 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:55.324910 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54156051-1272-461f-a7d5-c245d2257500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:55.324910 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54156051-1272-461f-a7d5-c245d2257500 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:55.324910 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54156051-1272-461f-a7d5-c245d2257500 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:55.324910 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54156051-1272-461f-a7d5-c245d2257500 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:55.422732 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1090/2191] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:46:55 2022] POST /identity/v3/users => generated 291 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:55.425450 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193c97f-070a-4050-995a-96f430aee33f admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:55.439818 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193c97f-070a-4050-995a-96f430aee33f admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:55.450995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d13fb1-7563-4ea4-b777-b609d209d41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:55.452159 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d13fb1-7563-4ea4-b777-b609d209d41d admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:55.452532 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d13fb1-7563-4ea4-b777-b609d209d41d admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:55.453056 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d13fb1-7563-4ea4-b777-b609d209d41d admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:55.454046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d13fb1-7563-4ea4-b777-b609d209d41d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:55.455781 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d13fb1-7563-4ea4-b777-b609d209d41d admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:55.473806 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1102/2192] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:55.478341 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54156051-1272-461f-a7d5-c245d2257500 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:55.503348 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54156051-1272-461f-a7d5-c245d2257500 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:55.510332 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f4343-d925-4db4-a22b-6013592df567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:55.511709 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f4343-d925-4db4-a22b-6013592df567 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:55.514142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f4343-d925-4db4-a22b-6013592df567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:55.514142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f4343-d925-4db4-a22b-6013592df567 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:55.514142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f4343-d925-4db4-a22b-6013592df567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:55.515808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f4343-d925-4db4-a22b-6013592df567 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:55.547271 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1091/2193] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:55.548461 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d13fb1-7563-4ea4-b777-b609d209d41d admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:55.571868 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d13fb1-7563-4ea4-b777-b609d209d41d admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:55.597961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7bc370-0f96-44df-8656-2808b783d1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:55.597961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bc370-0f96-44df-8656-2808b783d1ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:55.597961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bc370-0f96-44df-8656-2808b783d1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:55.597961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bc370-0f96-44df-8656-2808b783d1ab admin admin] PATH_INFO: `/v3/projects/cbfa353dd1cf43828f6209e995ee850a/users/9d467f42fd4a4e75ae27433601a8c702/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:55.645435 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7bc370-0f96-44df-8656-2808b783d1ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbfa353dd1cf43828f6209e995ee850a, user_id=9d467f42fd4a4e75ae27433601a8c702, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:55.649259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7bc370-0f96-44df-8656-2808b783d1ab admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:55.761126 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:55.766465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1103/2194] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:55 2022] PUT /identity/v3/projects/cbfa353dd1cf43828f6209e995ee850a/users/9d467f42fd4a4e75ae27433601a8c702/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:55.778937 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016b139-fc2b-45ea-bafa-2dbb39329941 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:55.780287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016b139-fc2b-45ea-bafa-2dbb39329941 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:55.780762 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016b139-fc2b-45ea-bafa-2dbb39329941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:55.839833 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-a016b139-fc2b-45ea-bafa-2dbb39329941 None None] MFA Rules not processed for user `9d467f42fd4a4e75ae27433601a8c702`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:55.843221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-a016b139-fc2b-45ea-bafa-2dbb39329941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:55.852875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:55.974540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1092/2195] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:55.978879 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7bc370-0f96-44df-8656-2808b783d1ab admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:56.070859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c7c55-a3e4-4766-9034-22819bfff429 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:56.071477 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c7c55-a3e4-4766-9034-22819bfff429 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:56.071764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c7c55-a3e4-4766-9034-22819bfff429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:56.092914 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7bc370-0f96-44df-8656-2808b783d1ab admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:56.098684 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4ca89-1d07-4cb9-8924-1d955024145a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:56.099643 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4ca89-1d07-4cb9-8924-1d955024145a admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:56.100744 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4ca89-1d07-4cb9-8924-1d955024145a admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:56.101223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4ca89-1d07-4cb9-8924-1d955024145a admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:56.102110 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4ca89-1d07-4cb9-8924-1d955024145a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:56.104370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4ca89-1d07-4cb9-8924-1d955024145a admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:56.109274 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-b59c7c55-a3e4-4766-9034-22819bfff429 None None] MFA Rules not processed for user `78b6599200a94a2a8bcc0149dc22944d`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:56.110938 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-b59c7c55-a3e4-4766-9034-22819bfff429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:56.196266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1093/2196] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:46:56.206424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703f760-cbd4-4c11-ae7b-cd4cf87b33f2 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:56.206904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703f760-cbd4-4c11-ae7b-cd4cf87b33f2 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:56.207349 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703f760-cbd4-4c11-ae7b-cd4cf87b33f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:56.226311 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-9703f760-cbd4-4c11-ae7b-cd4cf87b33f2 None None] MFA Rules not processed for user `78b6599200a94a2a8bcc0149dc22944d`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:56.227501 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-9703f760-cbd4-4c11-ae7b-cd4cf87b33f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:56.231823 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1104/2197] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:46:55 2022] POST /identity/v3/projects => generated 385 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:56.234692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4ca89-1d07-4cb9-8924-1d955024145a admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:56.291753 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1094/2198] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:46:56.304357 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703f760-cbd4-4c11-ae7b-cd4cf87b33f2 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:56.305916 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-9703f760-cbd4-4c11-ae7b-cd4cf87b33f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:56.307242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4ca89-1d07-4cb9-8924-1d955024145a admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:56.311958 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b8dc5-384b-4666-8810-960992739b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:56.312629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b8dc5-384b-4666-8810-960992739b95 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:56.313125 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b8dc5-384b-4666-8810-960992739b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:56.313406 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b8dc5-384b-4666-8810-960992739b95 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:56.314096 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b8dc5-384b-4666-8810-960992739b95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:56.316164 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b8dc5-384b-4666-8810-960992739b95 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:56.385810 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703f760-cbd4-4c11-ae7b-cd4cf87b33f2 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:56.391402 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1285fb4-9eb4-418b-9b5a-c74cc40be961 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:56.392075 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1285fb4-9eb4-418b-9b5a-c74cc40be961 service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:56.392358 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1285fb4-9eb4-418b-9b5a-c74cc40be961 service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:56.392652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1285fb4-9eb4-418b-9b5a-c74cc40be961 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:56.401296 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1285fb4-9eb4-418b-9b5a-c74cc40be961 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:56.408184 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1285fb4-9eb4-418b-9b5a-c74cc40be961 service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:56.411961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1095/2199] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:46:56.434174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1105/2200] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:46:56 2022] POST /identity/v3/users => generated 299 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:46:56.438470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1285fb4-9eb4-418b-9b5a-c74cc40be961 service nova] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:56.449849 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1285fb4-9eb4-418b-9b5a-c74cc40be961 service nova] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:56.454664 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65366d86-e5c3-4a5e-8532-37545d82637d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:56.455642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65366d86-e5c3-4a5e-8532-37545d82637d admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:56.455836 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65366d86-e5c3-4a5e-8532-37545d82637d admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:56.456118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65366d86-e5c3-4a5e-8532-37545d82637d admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:56.456910 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65366d86-e5c3-4a5e-8532-37545d82637d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:56.458514 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65366d86-e5c3-4a5e-8532-37545d82637d admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:56.473007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1096/2201] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:46:56.474066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b8dc5-384b-4666-8810-960992739b95 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:56.484809 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b8dc5-384b-4666-8810-960992739b95 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:56.489445 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b34bc-f729-4f24-ba3b-247e51aed634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:56.490866 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b34bc-f729-4f24-ba3b-247e51aed634 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:56.491180 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b34bc-f729-4f24-ba3b-247e51aed634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:56.491487 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b34bc-f729-4f24-ba3b-247e51aed634 admin admin] PATH_INFO: `/v3/projects/56fdf3bcaf004a75a97afd52f3ae6d54/users/63c86781dbe9474aaf2b5ae6817e6964/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:56.507497 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b34bc-f729-4f24-ba3b-247e51aed634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56fdf3bcaf004a75a97afd52f3ae6d54, user_id=63c86781dbe9474aaf2b5ae6817e6964, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:56.510452 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b34bc-f729-4f24-ba3b-247e51aed634 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:56.587568 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65366d86-e5c3-4a5e-8532-37545d82637d admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:56.588008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1106/2202] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:56 2022] PUT /identity/v3/projects/56fdf3bcaf004a75a97afd52f3ae6d54/users/63c86781dbe9474aaf2b5ae6817e6964/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:46:56.601975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65366d86-e5c3-4a5e-8532-37545d82637d admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:56.664813 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91a31d-a863-466b-a0fa-cb2a1fd00d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:56.665476 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91a31d-a863-466b-a0fa-cb2a1fd00d50 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:56.665757 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91a31d-a863-466b-a0fa-cb2a1fd00d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:56.666028 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91a31d-a863-466b-a0fa-cb2a1fd00d50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:56.666790 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91a31d-a863-466b-a0fa-cb2a1fd00d50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:56.668418 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91a31d-a863-466b-a0fa-cb2a1fd00d50 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:56.682728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1097/2203] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:46:56.683931 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b34bc-f729-4f24-ba3b-247e51aed634 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:56.696937 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b34bc-f729-4f24-ba3b-247e51aed634 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:56.703374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093c451-deb2-4474-bbf5-5b7041a593c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:56.704260 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093c451-deb2-4474-bbf5-5b7041a593c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:56.704590 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093c451-deb2-4474-bbf5-5b7041a593c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:56.705096 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093c451-deb2-4474-bbf5-5b7041a593c4 admin admin] PATH_INFO: `/v3/projects/56fdf3bcaf004a75a97afd52f3ae6d54/users/63c86781dbe9474aaf2b5ae6817e6964/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:56.707634 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9093c451-deb2-4474-bbf5-5b7041a593c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56fdf3bcaf004a75a97afd52f3ae6d54, user_id=63c86781dbe9474aaf2b5ae6817e6964, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:56.710618 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9093c451-deb2-4474-bbf5-5b7041a593c4 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:56.726435 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1107/2204] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:46:56 2022] PUT /identity/v3/projects/56fdf3bcaf004a75a97afd52f3ae6d54/users/63c86781dbe9474aaf2b5ae6817e6964/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:46:56.733509 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b9a2b-cbb4-4e00-934d-7555ca6f8a51 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:56.733799 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b9a2b-cbb4-4e00-934d-7555ca6f8a51 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:56.734079 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b9a2b-cbb4-4e00-934d-7555ca6f8a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:56.764800 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-225b9a2b-cbb4-4e00-934d-7555ca6f8a51 None None] MFA Rules not processed for user `63c86781dbe9474aaf2b5ae6817e6964`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:56.766007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-225b9a2b-cbb4-4e00-934d-7555ca6f8a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:56.871099 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1098/2205] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:56.877473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093c451-deb2-4474-bbf5-5b7041a593c4 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:46:56.879928 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-9093c451-deb2-4474-bbf5-5b7041a593c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:56.945695 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093c451-deb2-4474-bbf5-5b7041a593c4 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:46:56.950867 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169430d-8be1-49fe-b25f-d4c2e4b319ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:46:56.951516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169430d-8be1-49fe-b25f-d4c2e4b319ee service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:56.951798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169430d-8be1-49fe-b25f-d4c2e4b319ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:56.952073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169430d-8be1-49fe-b25f-d4c2e4b319ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:57.017941 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169430d-8be1-49fe-b25f-d4c2e4b319ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:46:57.019513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e169430d-8be1-49fe-b25f-d4c2e4b319ee service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:46:57.029520 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1108/2206] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:46:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:46:57.173686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e012e7-dbf9-4e98-b603-244419ca07e6 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:46:57.173686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e012e7-dbf9-4e98-b603-244419ca07e6 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:46:57.173686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e012e7-dbf9-4e98-b603-244419ca07e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:46:57.202060 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-d3e012e7-dbf9-4e98-b603-244419ca07e6 None None] MFA Rules not processed for user `78b6599200a94a2a8bcc0149dc22944d`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:46:57.202060 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-d3e012e7-dbf9-4e98-b603-244419ca07e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:46:57.286826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1099/2207] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:46:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:46:57.404393 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:57.612307 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:46:59.598335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:00.495477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:01.886582 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:01.945208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:02.152096 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169430d-8be1-49fe-b25f-d4c2e4b319ee service cinder] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:02.153453 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-e169430d-8be1-49fe-b25f-d4c2e4b319ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:02.249108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169430d-8be1-49fe-b25f-d4c2e4b319ee service cinder] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:02.254134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72898a07-1347-4fd3-9db9-4d7a468b2857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:02.255074 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72898a07-1347-4fd3-9db9-4d7a468b2857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:02.255556 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72898a07-1347-4fd3-9db9-4d7a468b2857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:02.256041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72898a07-1347-4fd3-9db9-4d7a468b2857 admin admin] PATH_INFO: `/v3/users/56d651e2aa7546f4ac7aec7150c119d3` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:02.257956 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72898a07-1347-4fd3-9db9-4d7a468b2857 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56d651e2aa7546f4ac7aec7150c119d3)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:02.260134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72898a07-1347-4fd3-9db9-4d7a468b2857 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:02.526194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-72898a07-1347-4fd3-9db9-4d7a468b2857 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56d651e2aa7546f4ac7aec7150c119d3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:02.533992 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-72898a07-1347-4fd3-9db9-4d7a468b2857 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56d651e2aa7546f4ac7aec7150c119d3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:02.534762 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-72898a07-1347-4fd3-9db9-4d7a468b2857 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56d651e2aa7546f4ac7aec7150c119d3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:02.535408 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-72898a07-1347-4fd3-9db9-4d7a468b2857 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56d651e2aa7546f4ac7aec7150c119d3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:02.548566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-72898a07-1347-4fd3-9db9-4d7a468b2857 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56d651e2aa7546f4ac7aec7150c119d3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:02.626756 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1109/2208] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:47:02 2022] DELETE /identity/v3/users/56d651e2aa7546f4ac7aec7150c119d3 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:47:02.634253 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2e1eb-2fa3-448d-9544-48c4ba714b40 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:02.634253 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2e1eb-2fa3-448d-9544-48c4ba714b40 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:02.634253 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2e1eb-2fa3-448d-9544-48c4ba714b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:02.655622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-2fa2e1eb-2fa3-448d-9544-48c4ba714b40 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:02.657569 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-2fa2e1eb-2fa3-448d-9544-48c4ba714b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:02.720111 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1100/2209] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:02.729825 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72898a07-1347-4fd3-9db9-4d7a468b2857 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:02.731635 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-72898a07-1347-4fd3-9db9-4d7a468b2857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:02.798910 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72898a07-1347-4fd3-9db9-4d7a468b2857 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:02.803476 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a6fb1-7c08-43c9-9858-fd6900abca1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:02.804215 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a6fb1-7c08-43c9-9858-fd6900abca1e service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:02.804496 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a6fb1-7c08-43c9-9858-fd6900abca1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:02.804772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a6fb1-7c08-43c9-9858-fd6900abca1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:02.818421 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a6fb1-7c08-43c9-9858-fd6900abca1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:02.820352 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a6fb1-7c08-43c9-9858-fd6900abca1e service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:02.836144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1110/2210] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:47:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:47:03.291082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2e1eb-2fa3-448d-9544-48c4ba714b40 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:03.292530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-2fa2e1eb-2fa3-448d-9544-48c4ba714b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:03.304534 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2e1eb-2fa3-448d-9544-48c4ba714b40 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:03.309355 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01849f8-8961-45b1-9325-d0015699eeca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:03.310698 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01849f8-8961-45b1-9325-d0015699eeca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:03.311606 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01849f8-8961-45b1-9325-d0015699eeca admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:03.311606 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01849f8-8961-45b1-9325-d0015699eeca admin admin] PATH_INFO: `/v3/projects/b53cd052002448e1bbb54e2b99271802` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:03.312488 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01849f8-8961-45b1-9325-d0015699eeca admin admin] RBAC: Authorizing `identity:delete_project(project_id=b53cd052002448e1bbb54e2b99271802)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:03.314443 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01849f8-8961-45b1-9325-d0015699eeca admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:03.343627 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:03.437396 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f01849f8-8961-45b1-9325-d0015699eeca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b53cd052002448e1bbb54e2b99271802', 'request_id': 'req-f01849f8-8961-45b1-9325-d0015699eeca'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:03.446213 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:03.519631 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f01849f8-8961-45b1-9325-d0015699eeca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b53cd052002448e1bbb54e2b99271802', 'request_id': 'req-f01849f8-8961-45b1-9325-d0015699eeca'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:03.526803 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f01849f8-8961-45b1-9325-d0015699eeca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b53cd052002448e1bbb54e2b99271802', 'request_id': 'req-f01849f8-8961-45b1-9325-d0015699eeca'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:03.537729 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f01849f8-8961-45b1-9325-d0015699eeca admin admin] The token cache is being invalidate because project b53cd052002448e1bbb54e2b99271802 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:47:03.537941 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f01849f8-8961-45b1-9325-d0015699eeca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:03.538611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f01849f8-8961-45b1-9325-d0015699eeca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:03.540155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1101/2211] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:47:03 2022] DELETE /identity/v3/projects/b53cd052002448e1bbb54e2b99271802 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:47:03.544339 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a6fb1-7c08-43c9-9858-fd6900abca1e service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:03.554699 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a6fb1-7c08-43c9-9858-fd6900abca1e service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:03.613294 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce34772-d0de-423b-8829-acb98eafae66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:03.614222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce34772-d0de-423b-8829-acb98eafae66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:03.614537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce34772-d0de-423b-8829-acb98eafae66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:03.614813 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce34772-d0de-423b-8829-acb98eafae66 admin admin] PATH_INFO: `/v3/users/d5c1215931f5413799f4c66171df330f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:03.616663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce34772-d0de-423b-8829-acb98eafae66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5c1215931f5413799f4c66171df330f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:03.618657 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce34772-d0de-423b-8829-acb98eafae66 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:03.817768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-2ce34772-d0de-423b-8829-acb98eafae66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5c1215931f5413799f4c66171df330f'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:03.826121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-2ce34772-d0de-423b-8829-acb98eafae66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5c1215931f5413799f4c66171df330f'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:03.826884 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-2ce34772-d0de-423b-8829-acb98eafae66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5c1215931f5413799f4c66171df330f'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:03.827501 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-2ce34772-d0de-423b-8829-acb98eafae66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5c1215931f5413799f4c66171df330f'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:03.841847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-2ce34772-d0de-423b-8829-acb98eafae66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5c1215931f5413799f4c66171df330f'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:03.926182 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1111/2212] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:47:03 2022] DELETE /identity/v3/users/d5c1215931f5413799f4c66171df330f => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:47:04.432249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:04.543301 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01849f8-8961-45b1-9325-d0015699eeca admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:04.544790 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f01849f8-8961-45b1-9325-d0015699eeca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:04.558149 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01849f8-8961-45b1-9325-d0015699eeca admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:04.621137 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f772b-ad4b-4c52-bff7-f2b46a4b4875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:04.622097 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f772b-ad4b-4c52-bff7-f2b46a4b4875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:04.622472 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f772b-ad4b-4c52-bff7-f2b46a4b4875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:04.622741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f772b-ad4b-4c52-bff7-f2b46a4b4875 admin admin] PATH_INFO: `/v3/projects/1fccda09b59148819bedc73cc32bdff3` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:04.627107 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f772b-ad4b-4c52-bff7-f2b46a4b4875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fccda09b59148819bedc73cc32bdff3)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:04.629212 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f772b-ad4b-4c52-bff7-f2b46a4b4875 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:04.713486 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:04.784310 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-5f2f772b-ad4b-4c52-bff7-f2b46a4b4875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fccda09b59148819bedc73cc32bdff3', 'request_id': 'req-5f2f772b-ad4b-4c52-bff7-f2b46a4b4875'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:04.894408 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-5f2f772b-ad4b-4c52-bff7-f2b46a4b4875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fccda09b59148819bedc73cc32bdff3', 'request_id': 'req-5f2f772b-ad4b-4c52-bff7-f2b46a4b4875'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:04.905996 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-5f2f772b-ad4b-4c52-bff7-f2b46a4b4875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fccda09b59148819bedc73cc32bdff3', 'request_id': 'req-5f2f772b-ad4b-4c52-bff7-f2b46a4b4875'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:04.928579 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-5f2f772b-ad4b-4c52-bff7-f2b46a4b4875 admin admin] The token cache is being invalidate because project 1fccda09b59148819bedc73cc32bdff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:47:04.929134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-5f2f772b-ad4b-4c52-bff7-f2b46a4b4875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:04.935781 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-5f2f772b-ad4b-4c52-bff7-f2b46a4b4875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:04.941530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1102/2213] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:47:04 2022] DELETE /identity/v3/projects/1fccda09b59148819bedc73cc32bdff3 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:47:04.942686 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24a1bc-a8c3-4ae2-a797-9242ad57fa98 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:04.943748 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24a1bc-a8c3-4ae2-a797-9242ad57fa98 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:04.944311 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24a1bc-a8c3-4ae2-a797-9242ad57fa98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:04.991416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-9c24a1bc-a8c3-4ae2-a797-9242ad57fa98 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:04.992649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-9c24a1bc-a8c3-4ae2-a797-9242ad57fa98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:05.106942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1112/2214] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:47:05.112379 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f772b-ad4b-4c52-bff7-f2b46a4b4875 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:05.112379 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-5f2f772b-ad4b-4c52-bff7-f2b46a4b4875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:05.120854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaf8af-63f1-4ea4-b9f4-2e992d5e93b9 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:05.121148 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaf8af-63f1-4ea4-b9f4-2e992d5e93b9 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:05.121484 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaf8af-63f1-4ea4-b9f4-2e992d5e93b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:05.172895 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-7edaf8af-63f1-4ea4-b9f4-2e992d5e93b9 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:05.174595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-7edaf8af-63f1-4ea4-b9f4-2e992d5e93b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:05.190420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1113/2215] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:47:05.192493 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edaf8af-63f1-4ea4-b9f4-2e992d5e93b9 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:05.209092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edaf8af-63f1-4ea4-b9f4-2e992d5e93b9 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:05.217339 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f772b-ad4b-4c52-bff7-f2b46a4b4875 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:05.217940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46a266a-24b9-4a30-a229-caafdf7acdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:05.219304 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a266a-24b9-4a30-a229-caafdf7acdc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:05.219882 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a266a-24b9-4a30-a229-caafdf7acdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:05.220310 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a266a-24b9-4a30-a229-caafdf7acdc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:05.221672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46a266a-24b9-4a30-a229-caafdf7acdc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:47:05.222149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46a266a-24b9-4a30-a229-caafdf7acdc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:05.222526 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9cf5dc-5539-467b-8d59-758cb8f3540e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:05.223893 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cf5dc-5539-467b-8d59-758cb8f3540e service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:05.224948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cf5dc-5539-467b-8d59-758cb8f3540e service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:05.225179 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46a266a-24b9-4a30-a229-caafdf7acdc3 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:05.225336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9cf5dc-5539-467b-8d59-758cb8f3540e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:05.228316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-3e9cf5dc-5539-467b-8d59-758cb8f3540e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:05.239689 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1114/2216] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:47:05.244576 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46a266a-24b9-4a30-a229-caafdf7acdc3 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:05.258106 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46a266a-24b9-4a30-a229-caafdf7acdc3 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:05.266338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e32a67-6343-49ce-a7d4-95c963162082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:05.267986 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e32a67-6343-49ce-a7d4-95c963162082 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:05.268349 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e32a67-6343-49ce-a7d4-95c963162082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:05.268697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e32a67-6343-49ce-a7d4-95c963162082 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:05.270562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e32a67-6343-49ce-a7d4-95c963162082 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:05.278486 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e32a67-6343-49ce-a7d4-95c963162082 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:05.324262 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9cf5dc-5539-467b-8d59-758cb8f3540e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:05.325776 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9cf5dc-5539-467b-8d59-758cb8f3540e service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:05.337254 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1103/2217] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:47:05 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:47:05.416160 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1115/2218] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:47:05 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:47:05.417162 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9cf5dc-5539-467b-8d59-758cb8f3540e service cinder] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:05.429460 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9cf5dc-5539-467b-8d59-758cb8f3540e service cinder] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:05.503203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520c51f-c63c-4280-89a5-8481bd7aa00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:05.504264 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520c51f-c63c-4280-89a5-8481bd7aa00d admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:05.504550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520c51f-c63c-4280-89a5-8481bd7aa00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:05.505339 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520c51f-c63c-4280-89a5-8481bd7aa00d admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:05.506566 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520c51f-c63c-4280-89a5-8481bd7aa00d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:05.508589 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520c51f-c63c-4280-89a5-8481bd7aa00d admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:05.590141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1104/2219] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:47:05 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:47:05.591375 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e32a67-6343-49ce-a7d4-95c963162082 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:05.602666 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e32a67-6343-49ce-a7d4-95c963162082 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:05.607594 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6b7e6-9d08-45bb-9c05-07b2fcc4c22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:05.608242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6b7e6-9d08-45bb-9c05-07b2fcc4c22b admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:05.608524 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6b7e6-9d08-45bb-9c05-07b2fcc4c22b admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:05.608801 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6b7e6-9d08-45bb-9c05-07b2fcc4c22b admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:05.609526 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6b7e6-9d08-45bb-9c05-07b2fcc4c22b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:05.611185 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6b7e6-9d08-45bb-9c05-07b2fcc4c22b admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:05.620134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:05.621461 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1116/2220] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:47:05.625250 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520c51f-c63c-4280-89a5-8481bd7aa00d admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:05.634977 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520c51f-c63c-4280-89a5-8481bd7aa00d admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:05.639486 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9cc22c-1f54-42db-aae4-ffa3bd9396eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:05.640130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9cc22c-1f54-42db-aae4-ffa3bd9396eb admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:05.640410 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9cc22c-1f54-42db-aae4-ffa3bd9396eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:05.640682 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9cc22c-1f54-42db-aae4-ffa3bd9396eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:05.641444 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9cc22c-1f54-42db-aae4-ffa3bd9396eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:05.643098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9cc22c-1f54-42db-aae4-ffa3bd9396eb admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:05.653405 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1105/2221] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:47:05 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:47:05.657578 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6b7e6-9d08-45bb-9c05-07b2fcc4c22b admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:05.668939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6b7e6-9d08-45bb-9c05-07b2fcc4c22b admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:05.673798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152b020-19f1-4d80-8c1c-85a4191ae0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:05.674702 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152b020-19f1-4d80-8c1c-85a4191ae0b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:05.674979 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152b020-19f1-4d80-8c1c-85a4191ae0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:05.675256 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152b020-19f1-4d80-8c1c-85a4191ae0b5 admin admin] PATH_INFO: `/v3/projects/6b3533b953874680bfd59cb3ce142d36/users/d0a9509d789347e2b0fdd717139b87be/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:05.691180 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152b020-19f1-4d80-8c1c-85a4191ae0b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b3533b953874680bfd59cb3ce142d36, user_id=d0a9509d789347e2b0fdd717139b87be, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:05.694839 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152b020-19f1-4d80-8c1c-85a4191ae0b5 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:05.811819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:05.821122 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1117/2222] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:47:05 2022] PUT /identity/v3/projects/6b3533b953874680bfd59cb3ce142d36/users/d0a9509d789347e2b0fdd717139b87be/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:47:05.831366 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc10f15-06b4-4f88-b54b-d379fd8a8e4a None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:05.831770 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc10f15-06b4-4f88-b54b-d379fd8a8e4a None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:05.832141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc10f15-06b4-4f88-b54b-d379fd8a8e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:05.859989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-0dc10f15-06b4-4f88-b54b-d379fd8a8e4a None None] MFA Rules not processed for user `d0a9509d789347e2b0fdd717139b87be`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:05.861249 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-0dc10f15-06b4-4f88-b54b-d379fd8a8e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:05.936505 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1106/2223] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:05.941064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152b020-19f1-4d80-8c1c-85a4191ae0b5 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:06.013626 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152b020-19f1-4d80-8c1c-85a4191ae0b5 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:06.034880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7dd94e-392b-4d33-971c-ccdd53c98bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:06.034880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7dd94e-392b-4d33-971c-ccdd53c98bed admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:06.034880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7dd94e-392b-4d33-971c-ccdd53c98bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:06.034880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7dd94e-392b-4d33-971c-ccdd53c98bed admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:06.035605 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7dd94e-392b-4d33-971c-ccdd53c98bed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:06.038090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7dd94e-392b-4d33-971c-ccdd53c98bed admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:06.136617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:06.143389 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1118/2224] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:47:05 2022] POST /identity/v3/projects => generated 385 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:47:06.151153 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc10f15-06b4-4f88-b54b-d379fd8a8e4a None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:06.167617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc10f15-06b4-4f88-b54b-d379fd8a8e4a None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:06.247398 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccea088b-2527-4408-a97a-75c8a8a85043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:06.248069 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea088b-2527-4408-a97a-75c8a8a85043 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:06.248356 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea088b-2527-4408-a97a-75c8a8a85043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:06.248624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea088b-2527-4408-a97a-75c8a8a85043 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:06.249314 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccea088b-2527-4408-a97a-75c8a8a85043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:06.251278 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccea088b-2527-4408-a97a-75c8a8a85043 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:06.412210 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1107/2225] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:47:06 2022] POST /identity/v3/users => generated 299 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:47:06.413357 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7dd94e-392b-4d33-971c-ccdd53c98bed admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:06.423752 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7dd94e-392b-4d33-971c-ccdd53c98bed admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:06.429074 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e3bbf-3877-4ea9-985c-60e6496492d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:06.429873 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e3bbf-3877-4ea9-985c-60e6496492d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:06.430162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e3bbf-3877-4ea9-985c-60e6496492d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:06.430479 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e3bbf-3877-4ea9-985c-60e6496492d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:06.431220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e3bbf-3877-4ea9-985c-60e6496492d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:06.432857 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e3bbf-3877-4ea9-985c-60e6496492d8 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:06.445135 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1119/2226] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:47:06.449457 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccea088b-2527-4408-a97a-75c8a8a85043 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:06.460762 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccea088b-2527-4408-a97a-75c8a8a85043 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:06.465627 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33633695-ba7e-4932-a1f4-e82d3b78d817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:06.466531 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33633695-ba7e-4932-a1f4-e82d3b78d817 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:06.466841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33633695-ba7e-4932-a1f4-e82d3b78d817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:06.467116 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33633695-ba7e-4932-a1f4-e82d3b78d817 admin admin] PATH_INFO: `/v3/projects/a03b3172a5c341cf8ded836bb8daecc6/users/b8d4ec8eff0c45ec9e3b351742d3b237/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:06.482887 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33633695-ba7e-4932-a1f4-e82d3b78d817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a03b3172a5c341cf8ded836bb8daecc6, user_id=b8d4ec8eff0c45ec9e3b351742d3b237, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:06.485571 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33633695-ba7e-4932-a1f4-e82d3b78d817 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:06.579155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1108/2227] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:47:06 2022] PUT /identity/v3/projects/a03b3172a5c341cf8ded836bb8daecc6/users/b8d4ec8eff0c45ec9e3b351742d3b237/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:47:06.580761 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e3bbf-3877-4ea9-985c-60e6496492d8 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:06.590817 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e3bbf-3877-4ea9-985c-60e6496492d8 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:06.647659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26960db9-1c97-43eb-8d01-bfa4883df3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:06.648390 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26960db9-1c97-43eb-8d01-bfa4883df3be admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:06.648605 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26960db9-1c97-43eb-8d01-bfa4883df3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:06.648885 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26960db9-1c97-43eb-8d01-bfa4883df3be admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:06.649619 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26960db9-1c97-43eb-8d01-bfa4883df3be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:06.651297 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26960db9-1c97-43eb-8d01-bfa4883df3be admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:06.663121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1120/2228] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:47:06.667368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33633695-ba7e-4932-a1f4-e82d3b78d817 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:06.679861 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33633695-ba7e-4932-a1f4-e82d3b78d817 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:06.684940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91225daa-220b-4ab5-ba4c-c1fda2424c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:06.686004 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91225daa-220b-4ab5-ba4c-c1fda2424c53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:06.686473 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91225daa-220b-4ab5-ba4c-c1fda2424c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:06.686893 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91225daa-220b-4ab5-ba4c-c1fda2424c53 admin admin] PATH_INFO: `/v3/projects/a03b3172a5c341cf8ded836bb8daecc6/users/b8d4ec8eff0c45ec9e3b351742d3b237/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:06.689251 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91225daa-220b-4ab5-ba4c-c1fda2424c53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a03b3172a5c341cf8ded836bb8daecc6, user_id=b8d4ec8eff0c45ec9e3b351742d3b237, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:06.692326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91225daa-220b-4ab5-ba4c-c1fda2424c53 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:06.725016 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1109/2229] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:47:06 2022] PUT /identity/v3/projects/a03b3172a5c341cf8ded836bb8daecc6/users/b8d4ec8eff0c45ec9e3b351742d3b237/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:47:06.738457 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd4ac9-8ae5-40a5-bb0d-2bfcf1402b33 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:06.738457 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd4ac9-8ae5-40a5-bb0d-2bfcf1402b33 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:06.738457 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd4ac9-8ae5-40a5-bb0d-2bfcf1402b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:06.774825 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-cbbd4ac9-8ae5-40a5-bb0d-2bfcf1402b33 None None] MFA Rules not processed for user `b8d4ec8eff0c45ec9e3b351742d3b237`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:06.776044 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-cbbd4ac9-8ae5-40a5-bb0d-2bfcf1402b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:06.809071 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:06.872469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1121/2230] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:06 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:06.883084 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91225daa-220b-4ab5-ba4c-c1fda2424c53 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:06.884889 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-91225daa-220b-4ab5-ba4c-c1fda2424c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:06.952794 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91225daa-220b-4ab5-ba4c-c1fda2424c53 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:06.956312 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9f3fc-fd03-436f-8af4-4834bad3349b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:06.956989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9f3fc-fd03-436f-8af4-4834bad3349b service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:06.957251 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9f3fc-fd03-436f-8af4-4834bad3349b service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:06.957522 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9f3fc-fd03-436f-8af4-4834bad3349b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:06.967100 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9f3fc-fd03-436f-8af4-4834bad3349b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:06.968651 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9f3fc-fd03-436f-8af4-4834bad3349b service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:06.979040 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1110/2231] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:47:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:47:07.566603 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:07.980992 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:08.195532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:08.858284 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:08.979063 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd4ac9-8ae5-40a5-bb0d-2bfcf1402b33 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:09.085405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd4ac9-8ae5-40a5-bb0d-2bfcf1402b33 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:09.090166 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e81df0-2077-4fa8-bd8b-bceda2197771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:09.091182 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e81df0-2077-4fa8-bd8b-bceda2197771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:09.091624 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e81df0-2077-4fa8-bd8b-bceda2197771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:09.091969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e81df0-2077-4fa8-bd8b-bceda2197771 admin admin] PATH_INFO: `/v3/users/d0a9509d789347e2b0fdd717139b87be` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:09.093159 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e81df0-2077-4fa8-bd8b-bceda2197771 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0a9509d789347e2b0fdd717139b87be)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:09.095152 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e81df0-2077-4fa8-bd8b-bceda2197771 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:09.286471 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b6e81df0-2077-4fa8-bd8b-bceda2197771 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0a9509d789347e2b0fdd717139b87be'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:09.293612 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b6e81df0-2077-4fa8-bd8b-bceda2197771 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0a9509d789347e2b0fdd717139b87be'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:09.294306 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b6e81df0-2077-4fa8-bd8b-bceda2197771 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0a9509d789347e2b0fdd717139b87be'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:09.294971 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b6e81df0-2077-4fa8-bd8b-bceda2197771 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0a9509d789347e2b0fdd717139b87be'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:09.307228 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b6e81df0-2077-4fa8-bd8b-bceda2197771 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0a9509d789347e2b0fdd717139b87be'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:09.402791 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1122/2232] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:47:08 2022] DELETE /identity/v3/users/d0a9509d789347e2b0fdd717139b87be => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:47:09.407418 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d2fc9-81f1-43f5-bc81-ff5474710ff6 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:09.408216 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d2fc9-81f1-43f5-bc81-ff5474710ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:09.408502 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d2fc9-81f1-43f5-bc81-ff5474710ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:09.435028 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-642d2fc9-81f1-43f5-bc81-ff5474710ff6 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:09.436231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-642d2fc9-81f1-43f5-bc81-ff5474710ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:09.496187 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1111/2233] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:09.506295 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e81df0-2077-4fa8-bd8b-bceda2197771 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:09.507732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-b6e81df0-2077-4fa8-bd8b-bceda2197771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:09.598017 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e81df0-2077-4fa8-bd8b-bceda2197771 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:09.602321 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a20cb3-1bfb-415e-9dba-f796e2353700 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:09.603043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a20cb3-1bfb-415e-9dba-f796e2353700 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:09.603329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a20cb3-1bfb-415e-9dba-f796e2353700 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:09.603607 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a20cb3-1bfb-415e-9dba-f796e2353700 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:09.612960 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a20cb3-1bfb-415e-9dba-f796e2353700 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:09.615817 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a20cb3-1bfb-415e-9dba-f796e2353700 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:09.624368 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1123/2234] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:47:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:47:10.022881 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d2fc9-81f1-43f5-bc81-ff5474710ff6 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:10.024628 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-642d2fc9-81f1-43f5-bc81-ff5474710ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:10.040247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d2fc9-81f1-43f5-bc81-ff5474710ff6 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:10.050483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33fa90-c1a7-4a68-bba9-7c61a44ff155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:10.050483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33fa90-c1a7-4a68-bba9-7c61a44ff155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:10.050483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33fa90-c1a7-4a68-bba9-7c61a44ff155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:10.050483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33fa90-c1a7-4a68-bba9-7c61a44ff155 admin admin] PATH_INFO: `/v3/projects/6b3533b953874680bfd59cb3ce142d36` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:10.051515 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c33fa90-c1a7-4a68-bba9-7c61a44ff155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b3533b953874680bfd59cb3ce142d36)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:10.053709 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c33fa90-c1a7-4a68-bba9-7c61a44ff155 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:10.191624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:10.234126 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-1c33fa90-c1a7-4a68-bba9-7c61a44ff155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b3533b953874680bfd59cb3ce142d36', 'request_id': 'req-1c33fa90-c1a7-4a68-bba9-7c61a44ff155'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:10.295562 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-1c33fa90-c1a7-4a68-bba9-7c61a44ff155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b3533b953874680bfd59cb3ce142d36', 'request_id': 'req-1c33fa90-c1a7-4a68-bba9-7c61a44ff155'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:10.310502 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-1c33fa90-c1a7-4a68-bba9-7c61a44ff155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b3533b953874680bfd59cb3ce142d36', 'request_id': 'req-1c33fa90-c1a7-4a68-bba9-7c61a44ff155'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:10.315562 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-1c33fa90-c1a7-4a68-bba9-7c61a44ff155 admin admin] The token cache is being invalidate because project 6b3533b953874680bfd59cb3ce142d36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:47:10.316030 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-1c33fa90-c1a7-4a68-bba9-7c61a44ff155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:10.316682 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-1c33fa90-c1a7-4a68-bba9-7c61a44ff155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:10.318547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1112/2235] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:47:10 2022] DELETE /identity/v3/projects/6b3533b953874680bfd59cb3ce142d36 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:47:10.322196 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a20cb3-1bfb-415e-9dba-f796e2353700 service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:10.333921 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a20cb3-1bfb-415e-9dba-f796e2353700 service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:10.397477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7befc1c-126d-41d3-bec9-d34946a1c6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:10.398540 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7befc1c-126d-41d3-bec9-d34946a1c6ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:10.398826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7befc1c-126d-41d3-bec9-d34946a1c6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:10.399102 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7befc1c-126d-41d3-bec9-d34946a1c6ab admin admin] PATH_INFO: `/v3/users/b8d4ec8eff0c45ec9e3b351742d3b237` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:10.400509 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7befc1c-126d-41d3-bec9-d34946a1c6ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8d4ec8eff0c45ec9e3b351742d3b237)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:10.402884 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7befc1c-126d-41d3-bec9-d34946a1c6ab admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:10.591879 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f7befc1c-126d-41d3-bec9-d34946a1c6ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8d4ec8eff0c45ec9e3b351742d3b237'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:10.600344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f7befc1c-126d-41d3-bec9-d34946a1c6ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8d4ec8eff0c45ec9e3b351742d3b237'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:10.601162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f7befc1c-126d-41d3-bec9-d34946a1c6ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8d4ec8eff0c45ec9e3b351742d3b237'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:10.601840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f7befc1c-126d-41d3-bec9-d34946a1c6ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8d4ec8eff0c45ec9e3b351742d3b237'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:10.616308 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f7befc1c-126d-41d3-bec9-d34946a1c6ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8d4ec8eff0c45ec9e3b351742d3b237'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:10.711726 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1124/2236] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:47:10 2022] DELETE /identity/v3/users/b8d4ec8eff0c45ec9e3b351742d3b237 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:47:10.734598 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:11.253856 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33fa90-c1a7-4a68-bba9-7c61a44ff155 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:11.257598 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-1c33fa90-c1a7-4a68-bba9-7c61a44ff155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:11.280595 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33fa90-c1a7-4a68-bba9-7c61a44ff155 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:11.360727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b17ea3-4370-4f3e-9085-47c1e9554ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:11.361619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b17ea3-4370-4f3e-9085-47c1e9554ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:11.361924 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b17ea3-4370-4f3e-9085-47c1e9554ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:11.362201 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b17ea3-4370-4f3e-9085-47c1e9554ff8 admin admin] PATH_INFO: `/v3/projects/a03b3172a5c341cf8ded836bb8daecc6` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:11.363480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b17ea3-4370-4f3e-9085-47c1e9554ff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a03b3172a5c341cf8ded836bb8daecc6)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:11.365729 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b17ea3-4370-4f3e-9085-47c1e9554ff8 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:11.479276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-96b17ea3-4370-4f3e-9085-47c1e9554ff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a03b3172a5c341cf8ded836bb8daecc6', 'request_id': 'req-96b17ea3-4370-4f3e-9085-47c1e9554ff8'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:11.584406 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-96b17ea3-4370-4f3e-9085-47c1e9554ff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a03b3172a5c341cf8ded836bb8daecc6', 'request_id': 'req-96b17ea3-4370-4f3e-9085-47c1e9554ff8'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:11.587322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-96b17ea3-4370-4f3e-9085-47c1e9554ff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a03b3172a5c341cf8ded836bb8daecc6', 'request_id': 'req-96b17ea3-4370-4f3e-9085-47c1e9554ff8'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:11.598563 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-96b17ea3-4370-4f3e-9085-47c1e9554ff8 admin admin] The token cache is being invalidate because project a03b3172a5c341cf8ded836bb8daecc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:47:11.598860 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-96b17ea3-4370-4f3e-9085-47c1e9554ff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:11.599906 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-96b17ea3-4370-4f3e-9085-47c1e9554ff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:11.601484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1113/2237] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:47:11 2022] DELETE /identity/v3/projects/a03b3172a5c341cf8ded836bb8daecc6 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:47:11.606626 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2f57d-4149-42bf-99fb-eebcec06488c None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:11.607079 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2f57d-4149-42bf-99fb-eebcec06488c None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:11.607195 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2f57d-4149-42bf-99fb-eebcec06488c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:11.651176 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-34a2f57d-4149-42bf-99fb-eebcec06488c None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:11.652721 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-34a2f57d-4149-42bf-99fb-eebcec06488c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:11.715196 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1125/2238] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:11.724269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319aa43b-d4e7-4d69-90a2-3b6b6457b32e None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:11.724559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319aa43b-d4e7-4d69-90a2-3b6b6457b32e None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:11.724842 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319aa43b-d4e7-4d69-90a2-3b6b6457b32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:11.747165 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-319aa43b-d4e7-4d69-90a2-3b6b6457b32e None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:11.748626 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-319aa43b-d4e7-4d69-90a2-3b6b6457b32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:11.763474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1114/2239] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:11.768509 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2f57d-4149-42bf-99fb-eebcec06488c None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:11.779311 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2f57d-4149-42bf-99fb-eebcec06488c None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:11.784483 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa063354-d737-4dc7-ae6c-ce9b573c4507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:11.785192 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa063354-d737-4dc7-ae6c-ce9b573c4507 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:11.785414 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa063354-d737-4dc7-ae6c-ce9b573c4507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:11.785693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa063354-d737-4dc7-ae6c-ce9b573c4507 admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:11.786481 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa063354-d737-4dc7-ae6c-ce9b573c4507 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:47:11.786795 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa063354-d737-4dc7-ae6c-ce9b573c4507 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:11.788565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa063354-d737-4dc7-ae6c-ce9b573c4507 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:11.805692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1126/2240] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:47:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:47:11.810288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319aa43b-d4e7-4d69-90a2-3b6b6457b32e None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:11.820450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319aa43b-d4e7-4d69-90a2-3b6b6457b32e None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:11.825470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d03ac-3a68-444a-9eae-3e77d21ed4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:11.826176 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d03ac-3a68-444a-9eae-3e77d21ed4f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:11.826540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d03ac-3a68-444a-9eae-3e77d21ed4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:11.826821 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d03ac-3a68-444a-9eae-3e77d21ed4f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:11.827522 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d03ac-3a68-444a-9eae-3e77d21ed4f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:11.829461 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d03ac-3a68-444a-9eae-3e77d21ed4f0 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:11.935212 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa063354-d737-4dc7-ae6c-ce9b573c4507 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:11.936366 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1115/2241] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:47:11 2022] POST /identity/v3/projects => generated 371 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:47:11.937244 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-fa063354-d737-4dc7-ae6c-ce9b573c4507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:11.942325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d03ac-3a68-444a-9eae-3e77d21ed4f0 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:11.953288 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa063354-d737-4dc7-ae6c-ce9b573c4507 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:12.018934 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d03ac-3a68-444a-9eae-3e77d21ed4f0 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:12.023063 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a432cee-5314-4fdf-b036-50bbf158bd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:12.023954 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a432cee-5314-4fdf-b036-50bbf158bd27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:12.025462 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce989a-61d1-49f9-accd-a3b41eafc03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:12.026078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a432cee-5314-4fdf-b036-50bbf158bd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:12.026078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a432cee-5314-4fdf-b036-50bbf158bd27 admin admin] PATH_INFO: `/v3/users/9d467f42fd4a4e75ae27433601a8c702` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:12.026266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce989a-61d1-49f9-accd-a3b41eafc03f admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:12.026475 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce989a-61d1-49f9-accd-a3b41eafc03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:12.026735 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce989a-61d1-49f9-accd-a3b41eafc03f admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:12.027426 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce989a-61d1-49f9-accd-a3b41eafc03f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:12.029465 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce989a-61d1-49f9-accd-a3b41eafc03f admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:12.030537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a432cee-5314-4fdf-b036-50bbf158bd27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d467f42fd4a4e75ae27433601a8c702)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:12.032566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a432cee-5314-4fdf-b036-50bbf158bd27 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:12.112203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1116/2242] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:47:11 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:47:12.116950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce989a-61d1-49f9-accd-a3b41eafc03f admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:12.128704 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce989a-61d1-49f9-accd-a3b41eafc03f admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:12.134197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb7a66-0344-4d23-99f1-8f019206b392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:12.135119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb7a66-0344-4d23-99f1-8f019206b392 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:12.135423 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb7a66-0344-4d23-99f1-8f019206b392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:12.135681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb7a66-0344-4d23-99f1-8f019206b392 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:12.136701 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb7a66-0344-4d23-99f1-8f019206b392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:12.138050 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb7a66-0344-4d23-99f1-8f019206b392 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:12.148686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1117/2243] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:47:12.153409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb7a66-0344-4d23-99f1-8f019206b392 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:12.165951 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb7a66-0344-4d23-99f1-8f019206b392 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:12.178284 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d0dd99-e966-4277-ab96-ef7e07902e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:12.178284 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0dd99-e966-4277-ab96-ef7e07902e09 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:12.178284 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0dd99-e966-4277-ab96-ef7e07902e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:12.178284 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0dd99-e966-4277-ab96-ef7e07902e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:12.178767 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0dd99-e966-4277-ab96-ef7e07902e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:12.178767 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0dd99-e966-4277-ab96-ef7e07902e09 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:12.186015 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1118/2244] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:47:12.190252 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d0dd99-e966-4277-ab96-ef7e07902e09 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:12.199920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d0dd99-e966-4277-ab96-ef7e07902e09 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:12.209652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e006707-411a-4213-854e-a7e7813768b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:12.209652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e006707-411a-4213-854e-a7e7813768b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:12.209652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e006707-411a-4213-854e-a7e7813768b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:12.209652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e006707-411a-4213-854e-a7e7813768b7 admin admin] PATH_INFO: `/v3/projects/4309afd9be5d4d1d8b61926ea675615a/users/cfd079ade9ff4a1d8a486178bc5ab242/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:12.229095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e006707-411a-4213-854e-a7e7813768b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4309afd9be5d4d1d8b61926ea675615a, user_id=cfd079ade9ff4a1d8a486178bc5ab242, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:12.232979 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e006707-411a-4213-854e-a7e7813768b7 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:12.247352 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7a432cee-5314-4fdf-b036-50bbf158bd27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d467f42fd4a4e75ae27433601a8c702'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:12.255531 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7a432cee-5314-4fdf-b036-50bbf158bd27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d467f42fd4a4e75ae27433601a8c702'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:12.258631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7a432cee-5314-4fdf-b036-50bbf158bd27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d467f42fd4a4e75ae27433601a8c702'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:12.258631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7a432cee-5314-4fdf-b036-50bbf158bd27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d467f42fd4a4e75ae27433601a8c702'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:12.273028 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7a432cee-5314-4fdf-b036-50bbf158bd27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d467f42fd4a4e75ae27433601a8c702'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:12.343655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1119/2245] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:47:12 2022] PUT /identity/v3/projects/4309afd9be5d4d1d8b61926ea675615a/users/cfd079ade9ff4a1d8a486178bc5ab242/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:47:12.354832 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a5327-96e3-464d-aa5d-ccec8773ba3c None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:12.355234 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a5327-96e3-464d-aa5d-ccec8773ba3c None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:12.355852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a5327-96e3-464d-aa5d-ccec8773ba3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:12.383248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1127/2246] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:47:11 2022] DELETE /identity/v3/users/9d467f42fd4a4e75ae27433601a8c702 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:47:12.387700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e1010-5759-4b5c-8d76-2771c4292d95 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:12.388102 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e1010-5759-4b5c-8d76-2771c4292d95 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:12.388484 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e1010-5759-4b5c-8d76-2771c4292d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:12.394360 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-6b5a5327-96e3-464d-aa5d-ccec8773ba3c None None] MFA Rules not processed for user `cfd079ade9ff4a1d8a486178bc5ab242`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:12.402480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-6b5a5327-96e3-464d-aa5d-ccec8773ba3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:12.419008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-538e1010-5759-4b5c-8d76-2771c4292d95 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:12.424623 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-538e1010-5759-4b5c-8d76-2771c4292d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:12.532760 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1120/2247] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:47:12.540929 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a5327-96e3-464d-aa5d-ccec8773ba3c None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:12.543220 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-6b5a5327-96e3-464d-aa5d-ccec8773ba3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:12.556034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1128/2248] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:47:12.559169 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e1010-5759-4b5c-8d76-2771c4292d95 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:12.563892 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-538e1010-5759-4b5c-8d76-2771c4292d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:12.635376 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:12.650265 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e1010-5759-4b5c-8d76-2771c4292d95 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:12.655271 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce143ffa-f0b1-4d67-a15d-cc833424ee3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:12.656229 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce143ffa-f0b1-4d67-a15d-cc833424ee3f service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:12.656891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce143ffa-f0b1-4d67-a15d-cc833424ee3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:12.657388 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce143ffa-f0b1-4d67-a15d-cc833424ee3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:12.667240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a5327-96e3-464d-aa5d-ccec8773ba3c None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:12.669256 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce143ffa-f0b1-4d67-a15d-cc833424ee3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:12.671232 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce143ffa-f0b1-4d67-a15d-cc833424ee3f service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:12.680216 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3a1bc-a317-40b4-a892-9e90a5c7a325 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:12.680216 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3a1bc-a317-40b4-a892-9e90a5c7a325 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:12.680216 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3a1bc-a317-40b4-a892-9e90a5c7a325 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:12.680216 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3a1bc-a317-40b4-a892-9e90a5c7a325 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:12.687375 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3a1bc-a317-40b4-a892-9e90a5c7a325 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:12.688002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3a1bc-a317-40b4-a892-9e90a5c7a325 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:12.692196 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1129/2249] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:47:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:47:12.720474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1121/2250] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:47:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:47:12.723906 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce143ffa-f0b1-4d67-a15d-cc833424ee3f service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:12.725514 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ce143ffa-f0b1-4d67-a15d-cc833424ee3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:12.778087 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce143ffa-f0b1-4d67-a15d-cc833424ee3f service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:12.804850 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df90ba-30f2-4302-9eed-a34963a8914e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:12.806161 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df90ba-30f2-4302-9eed-a34963a8914e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:12.806745 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df90ba-30f2-4302-9eed-a34963a8914e admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:12.809263 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df90ba-30f2-4302-9eed-a34963a8914e admin admin] PATH_INFO: `/v3/users/cfd079ade9ff4a1d8a486178bc5ab242` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:12.811195 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55df90ba-30f2-4302-9eed-a34963a8914e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfd079ade9ff4a1d8a486178bc5ab242)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:12.814173 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55df90ba-30f2-4302-9eed-a34963a8914e admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:12.862271 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81891f-885b-402b-aff0-618060093f02 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:12.862654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81891f-885b-402b-aff0-618060093f02 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:12.862987 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81891f-885b-402b-aff0-618060093f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:12.895958 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-2b81891f-885b-402b-aff0-618060093f02 None None] MFA Rules not processed for user `58e1f1ac6ea14da0be6e9a24d007847a`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:12.897485 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-2b81891f-885b-402b-aff0-618060093f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:12.965418 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1122/2251] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Jun 27 18:47:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:47:13.015584 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81891f-885b-402b-aff0-618060093f02 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:13.017089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-2b81891f-885b-402b-aff0-618060093f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:13.092302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-55df90ba-30f2-4302-9eed-a34963a8914e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfd079ade9ff4a1d8a486178bc5ab242'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:13.108203 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-55df90ba-30f2-4302-9eed-a34963a8914e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfd079ade9ff4a1d8a486178bc5ab242'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:13.111030 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-55df90ba-30f2-4302-9eed-a34963a8914e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfd079ade9ff4a1d8a486178bc5ab242'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:13.116706 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-55df90ba-30f2-4302-9eed-a34963a8914e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfd079ade9ff4a1d8a486178bc5ab242'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:13.123881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:13.134101 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81891f-885b-402b-aff0-618060093f02 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:13.140890 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe76fd-eea4-4aba-bc1a-79b473bc5de0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1529c9a45434a19968e924797688f06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:13.141748 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe76fd-eea4-4aba-bc1a-79b473bc5de0 service swift] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:13.142301 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe76fd-eea4-4aba-bc1a-79b473bc5de0 service swift] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:13.142884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe76fd-eea4-4aba-bc1a-79b473bc5de0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:13.145356 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-4efe76fd-eea4-4aba-bc1a-79b473bc5de0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:13.146847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-55df90ba-30f2-4302-9eed-a34963a8914e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfd079ade9ff4a1d8a486178bc5ab242'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:13.160406 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe76fd-eea4-4aba-bc1a-79b473bc5de0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:13.162163 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe76fd-eea4-4aba-bc1a-79b473bc5de0 service swift] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:13.176299 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1123/2252] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Jun 27 18:47:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 18:47:13.265854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1130/2253] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:47:12 2022] DELETE /identity/v3/users/cfd079ade9ff4a1d8a486178bc5ab242 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:47:13.271367 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0ff71-530d-470d-a341-c05fe95ed8f3 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:13.271367 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0ff71-530d-470d-a341-c05fe95ed8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:13.272476 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0ff71-530d-470d-a341-c05fe95ed8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:13.319764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-5ef0ff71-530d-470d-a341-c05fe95ed8f3 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:13.322645 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-5ef0ff71-530d-470d-a341-c05fe95ed8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:13.427537 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1124/2254] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:13.434647 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df90ba-30f2-4302-9eed-a34963a8914e admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:13.436355 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-55df90ba-30f2-4302-9eed-a34963a8914e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:13.455200 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0ff71-530d-470d-a341-c05fe95ed8f3 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:13.457593 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-5ef0ff71-530d-470d-a341-c05fe95ed8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:13.476123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0ff71-530d-470d-a341-c05fe95ed8f3 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:13.485174 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d03e8-3e91-4eac-858b-a97307b036b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:13.506714 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d03e8-3e91-4eac-858b-a97307b036b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:13.507023 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d03e8-3e91-4eac-858b-a97307b036b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:13.507304 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d03e8-3e91-4eac-858b-a97307b036b8 admin admin] PATH_INFO: `/v3/projects/cbfa353dd1cf43828f6209e995ee850a` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:13.516313 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d03e8-3e91-4eac-858b-a97307b036b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbfa353dd1cf43828f6209e995ee850a)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:13.516313 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d03e8-3e91-4eac-858b-a97307b036b8 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:13.548118 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df90ba-30f2-4302-9eed-a34963a8914e admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:13.553891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c5d8d1-bb34-4475-b0b6-329a18cb4ca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:13.554785 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5d8d1-bb34-4475-b0b6-329a18cb4ca9 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:13.555073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5d8d1-bb34-4475-b0b6-329a18cb4ca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:13.555352 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5d8d1-bb34-4475-b0b6-329a18cb4ca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:13.568248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c5d8d1-bb34-4475-b0b6-329a18cb4ca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:13.569943 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c5d8d1-bb34-4475-b0b6-329a18cb4ca9 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:13.581185 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1131/2255] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:47:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:47:13.716265 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-d31d03e8-3e91-4eac-858b-a97307b036b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbfa353dd1cf43828f6209e995ee850a', 'request_id': 'req-d31d03e8-3e91-4eac-858b-a97307b036b8'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:13.843860 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-d31d03e8-3e91-4eac-858b-a97307b036b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbfa353dd1cf43828f6209e995ee850a', 'request_id': 'req-d31d03e8-3e91-4eac-858b-a97307b036b8'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:13.852664 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-d31d03e8-3e91-4eac-858b-a97307b036b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbfa353dd1cf43828f6209e995ee850a', 'request_id': 'req-d31d03e8-3e91-4eac-858b-a97307b036b8'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:13.873819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-d31d03e8-3e91-4eac-858b-a97307b036b8 admin admin] The token cache is being invalidate because project cbfa353dd1cf43828f6209e995ee850a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:47:13.875064 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-d31d03e8-3e91-4eac-858b-a97307b036b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:13.875940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-d31d03e8-3e91-4eac-858b-a97307b036b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:13.878285 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1125/2256] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:47:13 2022] DELETE /identity/v3/projects/cbfa353dd1cf43828f6209e995ee850a => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:47:13.882011 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c5d8d1-bb34-4475-b0b6-329a18cb4ca9 service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:13.892468 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c5d8d1-bb34-4475-b0b6-329a18cb4ca9 service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:14.015159 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59ca3a-18be-4283-8403-06681c79040b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:14.016518 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59ca3a-18be-4283-8403-06681c79040b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:14.016956 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59ca3a-18be-4283-8403-06681c79040b admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:14.017415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59ca3a-18be-4283-8403-06681c79040b admin admin] PATH_INFO: `/v3/users/63c86781dbe9474aaf2b5ae6817e6964` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:14.018972 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59ca3a-18be-4283-8403-06681c79040b admin admin] RBAC: Authorizing `identity:delete_user(user_id=63c86781dbe9474aaf2b5ae6817e6964)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:14.021899 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59ca3a-18be-4283-8403-06681c79040b admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:14.249774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d03e8-3e91-4eac-858b-a97307b036b8 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:14.249774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-d31d03e8-3e91-4eac-858b-a97307b036b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:14.259773 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d03e8-3e91-4eac-858b-a97307b036b8 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:14.307304 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f41b5-74f2-4712-b441-6b288c31227b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:14.308323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f41b5-74f2-4712-b441-6b288c31227b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:14.308656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f41b5-74f2-4712-b441-6b288c31227b admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:14.308932 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f41b5-74f2-4712-b441-6b288c31227b admin admin] PATH_INFO: `/v3/projects/4309afd9be5d4d1d8b61926ea675615a` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:14.310296 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f41b5-74f2-4712-b441-6b288c31227b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4309afd9be5d4d1d8b61926ea675615a)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:14.313440 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f41b5-74f2-4712-b441-6b288c31227b admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:14.839924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-2a59ca3a-18be-4283-8403-06681c79040b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63c86781dbe9474aaf2b5ae6817e6964'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:14.854115 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-2a59ca3a-18be-4283-8403-06681c79040b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63c86781dbe9474aaf2b5ae6817e6964'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:14.855132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-2a59ca3a-18be-4283-8403-06681c79040b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63c86781dbe9474aaf2b5ae6817e6964'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:14.856202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-2a59ca3a-18be-4283-8403-06681c79040b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63c86781dbe9474aaf2b5ae6817e6964'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:14.875830 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-618f41b5-74f2-4712-b441-6b288c31227b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4309afd9be5d4d1d8b61926ea675615a', 'request_id': 'req-618f41b5-74f2-4712-b441-6b288c31227b'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:14.881122 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-2a59ca3a-18be-4283-8403-06681c79040b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63c86781dbe9474aaf2b5ae6817e6964'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:14.970935 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-618f41b5-74f2-4712-b441-6b288c31227b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4309afd9be5d4d1d8b61926ea675615a', 'request_id': 'req-618f41b5-74f2-4712-b441-6b288c31227b'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:14.979665 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-618f41b5-74f2-4712-b441-6b288c31227b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4309afd9be5d4d1d8b61926ea675615a', 'request_id': 'req-618f41b5-74f2-4712-b441-6b288c31227b'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:14.993913 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-618f41b5-74f2-4712-b441-6b288c31227b admin admin] The token cache is being invalidate because project 4309afd9be5d4d1d8b61926ea675615a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:47:14.994360 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-618f41b5-74f2-4712-b441-6b288c31227b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:14.995902 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-618f41b5-74f2-4712-b441-6b288c31227b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:14.998433 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1126/2257] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:47:14 2022] DELETE /identity/v3/projects/4309afd9be5d4d1d8b61926ea675615a => generated 0 bytes in 768 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:47:15.003705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1f330-fc4d-4925-8af3-01ae85bb0d15 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:15.003997 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1f330-fc4d-4925-8af3-01ae85bb0d15 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:15.004312 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1f330-fc4d-4925-8af3-01ae85bb0d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:15.026970 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1132/2258] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:47:13 2022] DELETE /identity/v3/users/63c86781dbe9474aaf2b5ae6817e6964 => generated 0 bytes in 1145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:47:15.028599 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-3ce1f330-fc4d-4925-8af3-01ae85bb0d15 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:15.035543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-3ce1f330-fc4d-4925-8af3-01ae85bb0d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:15.112121 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1127/2259] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:47:15.121373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79056883-37ea-484b-8a86-9f71fe9cbfd5 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:15.121853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79056883-37ea-484b-8a86-9f71fe9cbfd5 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:15.121853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79056883-37ea-484b-8a86-9f71fe9cbfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:15.151275 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-79056883-37ea-484b-8a86-9f71fe9cbfd5 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:15.152752 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-79056883-37ea-484b-8a86-9f71fe9cbfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:15.171510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1133/2260] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:15.173382 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1f330-fc4d-4925-8af3-01ae85bb0d15 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:15.186881 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1f330-fc4d-4925-8af3-01ae85bb0d15 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:15.193329 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71cda88-59e8-400a-b1ae-187b9e25cdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:15.194085 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cda88-59e8-400a-b1ae-187b9e25cdc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:15.194514 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cda88-59e8-400a-b1ae-187b9e25cdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:15.194864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cda88-59e8-400a-b1ae-187b9e25cdc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:15.195828 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71cda88-59e8-400a-b1ae-187b9e25cdc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:47:15.197024 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71cda88-59e8-400a-b1ae-187b9e25cdc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:15.201883 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71cda88-59e8-400a-b1ae-187b9e25cdc3 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:15.218542 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1128/2261] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:47:15.223970 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79056883-37ea-484b-8a86-9f71fe9cbfd5 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:15.235381 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79056883-37ea-484b-8a86-9f71fe9cbfd5 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:15.240422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc711d5-1dde-4813-ac76-39430f755bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:15.241067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc711d5-1dde-4813-ac76-39430f755bdf admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:15.241351 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc711d5-1dde-4813-ac76-39430f755bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:15.241658 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc711d5-1dde-4813-ac76-39430f755bdf admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:15.242362 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc711d5-1dde-4813-ac76-39430f755bdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:15.244373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc711d5-1dde-4813-ac76-39430f755bdf admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:15.285294 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:15.362914 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1134/2262] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:47:15 2022] POST /identity/v3/projects => generated 377 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:47:15.369042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71cda88-59e8-400a-b1ae-187b9e25cdc3 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:15.385963 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71cda88-59e8-400a-b1ae-187b9e25cdc3 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:15.484831 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57309d-0d5b-4cdf-ae1f-010f723c8595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:15.485810 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57309d-0d5b-4cdf-ae1f-010f723c8595 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:15.486101 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57309d-0d5b-4cdf-ae1f-010f723c8595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:15.486393 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57309d-0d5b-4cdf-ae1f-010f723c8595 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:15.487102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57309d-0d5b-4cdf-ae1f-010f723c8595 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:15.495094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57309d-0d5b-4cdf-ae1f-010f723c8595 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:15.511465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:15.646317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1129/2263] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:47:15 2022] POST /identity/v3/users => generated 289 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:47:15.649700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc711d5-1dde-4813-ac76-39430f755bdf admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:15.655334 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57309d-0d5b-4cdf-ae1f-010f723c8595 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:15.664879 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-fa57309d-0d5b-4cdf-ae1f-010f723c8595 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:15.667571 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc711d5-1dde-4813-ac76-39430f755bdf admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:15.689816 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57309d-0d5b-4cdf-ae1f-010f723c8595 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:15.695952 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5588399-8f21-4f1d-8109-06d284cfbbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:15.696893 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5588399-8f21-4f1d-8109-06d284cfbbbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:15.697187 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5588399-8f21-4f1d-8109-06d284cfbbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:15.697625 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5588399-8f21-4f1d-8109-06d284cfbbbd admin admin] PATH_INFO: `/v3/projects/56fdf3bcaf004a75a97afd52f3ae6d54` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:15.700073 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5588399-8f21-4f1d-8109-06d284cfbbbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=56fdf3bcaf004a75a97afd52f3ae6d54)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:15.701550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5588399-8f21-4f1d-8109-06d284cfbbbd admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:15.702241 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b1887-f854-44b3-9aa1-bb3003ba4082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:15.711247 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b1887-f854-44b3-9aa1-bb3003ba4082 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:15.713318 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b1887-f854-44b3-9aa1-bb3003ba4082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:15.713318 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b1887-f854-44b3-9aa1-bb3003ba4082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:15.717039 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b1887-f854-44b3-9aa1-bb3003ba4082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:15.720938 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b1887-f854-44b3-9aa1-bb3003ba4082 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:15.745932 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1135/2264] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:47:15.749665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b1887-f854-44b3-9aa1-bb3003ba4082 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:15.764650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b1887-f854-44b3-9aa1-bb3003ba4082 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:15.769872 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626e16a-151c-4713-b0b3-fdf5a58b585a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:15.776379 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626e16a-151c-4713-b0b3-fdf5a58b585a admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:15.776379 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626e16a-151c-4713-b0b3-fdf5a58b585a admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:15.776379 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626e16a-151c-4713-b0b3-fdf5a58b585a admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:15.776379 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626e16a-151c-4713-b0b3-fdf5a58b585a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:15.776379 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626e16a-151c-4713-b0b3-fdf5a58b585a admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:15.787024 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1136/2265] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:47:15.796492 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626e16a-151c-4713-b0b3-fdf5a58b585a admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:15.813477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626e16a-151c-4713-b0b3-fdf5a58b585a admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:15.820805 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacde1c-98f3-477b-b660-eb8fe72ca180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:15.820805 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacde1c-98f3-477b-b660-eb8fe72ca180 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:15.821099 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacde1c-98f3-477b-b660-eb8fe72ca180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:15.821380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacde1c-98f3-477b-b660-eb8fe72ca180 admin admin] PATH_INFO: `/v3/projects/dafa445887284fe7a018c479dd8335cb/users/745099adc24a4b12b3e22cfadd29bd70/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:15.857372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bacde1c-98f3-477b-b660-eb8fe72ca180 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dafa445887284fe7a018c479dd8335cb, user_id=745099adc24a4b12b3e22cfadd29bd70, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:15.862697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bacde1c-98f3-477b-b660-eb8fe72ca180 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:15.869241 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f5588399-8f21-4f1d-8109-06d284cfbbbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56fdf3bcaf004a75a97afd52f3ae6d54', 'request_id': 'req-f5588399-8f21-4f1d-8109-06d284cfbbbd'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:16.003717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f5588399-8f21-4f1d-8109-06d284cfbbbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56fdf3bcaf004a75a97afd52f3ae6d54', 'request_id': 'req-f5588399-8f21-4f1d-8109-06d284cfbbbd'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:16.012079 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1137/2266] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:47:15 2022] PUT /identity/v3/projects/dafa445887284fe7a018c479dd8335cb/users/745099adc24a4b12b3e22cfadd29bd70/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:47:16.012427 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f5588399-8f21-4f1d-8109-06d284cfbbbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56fdf3bcaf004a75a97afd52f3ae6d54', 'request_id': 'req-f5588399-8f21-4f1d-8109-06d284cfbbbd'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:16.028176 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd0863-7f9b-44e3-8369-4b18e7d3f227 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:16.028743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd0863-7f9b-44e3-8369-4b18e7d3f227 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:16.028743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd0863-7f9b-44e3-8369-4b18e7d3f227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:16.069525 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f5588399-8f21-4f1d-8109-06d284cfbbbd admin admin] The token cache is being invalidate because project 56fdf3bcaf004a75a97afd52f3ae6d54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:47:16.069946 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f5588399-8f21-4f1d-8109-06d284cfbbbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:16.072377 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f5588399-8f21-4f1d-8109-06d284cfbbbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:16.076206 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1130/2267] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:47:15 2022] DELETE /identity/v3/projects/56fdf3bcaf004a75a97afd52f3ae6d54 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:47:16.078334 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:16.082125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75fc90-024b-4b72-8bd0-6d46727e47af None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:16.085375 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75fc90-024b-4b72-8bd0-6d46727e47af None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:16.085375 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75fc90-024b-4b72-8bd0-6d46727e47af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:16.093883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-93dd0863-7f9b-44e3-8369-4b18e7d3f227 None None] MFA Rules not processed for user `745099adc24a4b12b3e22cfadd29bd70`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:16.096118 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-93dd0863-7f9b-44e3-8369-4b18e7d3f227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:16.121605 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-5d75fc90-024b-4b72-8bd0-6d46727e47af None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:16.123664 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-5d75fc90-024b-4b72-8bd0-6d46727e47af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:16.212474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1131/2268] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:16.224179 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f422d6c-b1ca-4da1-aa32-eebb9fc8f0b8 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:16.228849 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f422d6c-b1ca-4da1-aa32-eebb9fc8f0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:16.228849 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f422d6c-b1ca-4da1-aa32-eebb9fc8f0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:16.253736 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-7f422d6c-b1ca-4da1-aa32-eebb9fc8f0b8 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:16.255518 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-7f422d6c-b1ca-4da1-aa32-eebb9fc8f0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:16.258771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1138/2269] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:47:16.271051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1132/2270] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:16.278773 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd0863-7f9b-44e3-8369-4b18e7d3f227 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:16.298778 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd0863-7f9b-44e3-8369-4b18e7d3f227 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:16.306809 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5386d06-38b1-4f34-913b-1a698e9fd083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:16.307801 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5386d06-38b1-4f34-913b-1a698e9fd083 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:16.308249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5386d06-38b1-4f34-913b-1a698e9fd083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:16.308695 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5386d06-38b1-4f34-913b-1a698e9fd083 admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:16.309445 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5386d06-38b1-4f34-913b-1a698e9fd083 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:47:16.309755 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5386d06-38b1-4f34-913b-1a698e9fd083 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:16.311402 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5386d06-38b1-4f34-913b-1a698e9fd083 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:16.325693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1139/2271] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:47:16.331998 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f422d6c-b1ca-4da1-aa32-eebb9fc8f0b8 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:16.347980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f422d6c-b1ca-4da1-aa32-eebb9fc8f0b8 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:16.354887 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c95dd9-dd44-467d-b6a1-46d581602d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:16.355912 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c95dd9-dd44-467d-b6a1-46d581602d98 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:16.356261 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c95dd9-dd44-467d-b6a1-46d581602d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:16.356605 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c95dd9-dd44-467d-b6a1-46d581602d98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:16.357395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c95dd9-dd44-467d-b6a1-46d581602d98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:16.359411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c95dd9-dd44-467d-b6a1-46d581602d98 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:16.506846 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1133/2272] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:47:16 2022] POST /identity/v3/projects => generated 373 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:47:16.508141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5386d06-38b1-4f34-913b-1a698e9fd083 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:16.540404 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5386d06-38b1-4f34-913b-1a698e9fd083 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:16.540404 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:16.617139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a092190-6543-489d-9025-341b6c108460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:16.617660 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a092190-6543-489d-9025-341b6c108460 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:16.618038 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a092190-6543-489d-9025-341b6c108460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:16.618426 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a092190-6543-489d-9025-341b6c108460 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:16.619218 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a092190-6543-489d-9025-341b6c108460 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:16.621672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a092190-6543-489d-9025-341b6c108460 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:16.635750 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:16.771342 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1140/2273] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:47:16 2022] POST /identity/v3/users => generated 287 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:47:16.773216 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c95dd9-dd44-467d-b6a1-46d581602d98 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:16.789617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c95dd9-dd44-467d-b6a1-46d581602d98 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:16.796840 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b980ba-0773-4ded-97c6-729522312541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:16.798005 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b980ba-0773-4ded-97c6-729522312541 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:16.798605 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b980ba-0773-4ded-97c6-729522312541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:16.799026 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b980ba-0773-4ded-97c6-729522312541 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:16.801382 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b980ba-0773-4ded-97c6-729522312541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:16.804407 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b980ba-0773-4ded-97c6-729522312541 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:16.821755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1134/2274] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:47:16.850738 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a092190-6543-489d-9025-341b6c108460 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:16.850738 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a092190-6543-489d-9025-341b6c108460 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:16.850738 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9fc57-cff9-4845-8c70-46be3618937f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:16.850738 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9fc57-cff9-4845-8c70-46be3618937f admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:16.851898 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9fc57-cff9-4845-8c70-46be3618937f admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:16.851898 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9fc57-cff9-4845-8c70-46be3618937f admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:16.851898 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9fc57-cff9-4845-8c70-46be3618937f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:16.851898 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9fc57-cff9-4845-8c70-46be3618937f admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:16.863555 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1141/2275] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:47:16.870032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b980ba-0773-4ded-97c6-729522312541 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:16.884430 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b980ba-0773-4ded-97c6-729522312541 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:16.889207 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6108e0-d5a7-46ba-aa37-19e2768de041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:16.890164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6108e0-d5a7-46ba-aa37-19e2768de041 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:16.890578 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6108e0-d5a7-46ba-aa37-19e2768de041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:16.890927 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6108e0-d5a7-46ba-aa37-19e2768de041 admin admin] PATH_INFO: `/v3/projects/0f87955d76bd4456b0b696a18b89b902/users/aa78b9c2e6f74103b61de91798b43b25/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:16.914769 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6108e0-d5a7-46ba-aa37-19e2768de041 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f87955d76bd4456b0b696a18b89b902, user_id=aa78b9c2e6f74103b61de91798b43b25, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:16.919861 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6108e0-d5a7-46ba-aa37-19e2768de041 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:17.241093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1135/2276] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:47:16 2022] PUT /identity/v3/projects/0f87955d76bd4456b0b696a18b89b902/users/aa78b9c2e6f74103b61de91798b43b25/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:47:17.253294 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e6fd2-8fd9-4980-b4f8-51ee0804656e None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:17.253821 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e6fd2-8fd9-4980-b4f8-51ee0804656e None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:17.253925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e6fd2-8fd9-4980-b4f8-51ee0804656e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:17.296167 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-de1e6fd2-8fd9-4980-b4f8-51ee0804656e None None] MFA Rules not processed for user `aa78b9c2e6f74103b61de91798b43b25`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:17.297398 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-de1e6fd2-8fd9-4980-b4f8-51ee0804656e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:17.395849 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:17.468774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1142/2277] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:17.486508 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6108e0-d5a7-46ba-aa37-19e2768de041 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:17.491353 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-fc6108e0-d5a7-46ba-aa37-19e2768de041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:17.567554 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:17.605618 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6108e0-d5a7-46ba-aa37-19e2768de041 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:17.612831 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22e01d-5d10-4b74-ab5e-ac4a4db10660 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:17.622733 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22e01d-5d10-4b74-ab5e-ac4a4db10660 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:17.626122 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22e01d-5d10-4b74-ab5e-ac4a4db10660 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:17.626122 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22e01d-5d10-4b74-ab5e-ac4a4db10660 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:17.645142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22e01d-5d10-4b74-ab5e-ac4a4db10660 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:17.658118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22e01d-5d10-4b74-ab5e-ac4a4db10660 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:17.681900 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1136/2278] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:47:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:47:17.734598 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:21.416402 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e6fd2-8fd9-4980-b4f8-51ee0804656e None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:21.418121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-de1e6fd2-8fd9-4980-b4f8-51ee0804656e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:21.537756 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e6fd2-8fd9-4980-b4f8-51ee0804656e None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:21.543494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676adc06-fe4b-4c6f-b883-97f87aae0857 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e75e5ae3ca45d0b503f6e52f399558', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:21.544678 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676adc06-fe4b-4c6f-b883-97f87aae0857 service glance] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:21.544678 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676adc06-fe4b-4c6f-b883-97f87aae0857 service glance] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:21.544871 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676adc06-fe4b-4c6f-b883-97f87aae0857 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:21.647843 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676adc06-fe4b-4c6f-b883-97f87aae0857 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:21.649319 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676adc06-fe4b-4c6f-b883-97f87aae0857 service glance] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:21.668206 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1143/2279] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:47:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:47:27.765935 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=34,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:31.655390 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=72 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:34.272928 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29e0dd-db5a-4557-a33b-e33a1d0a2c67 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:34.273530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29e0dd-db5a-4557-a33b-e33a1d0a2c67 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:34.273530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29e0dd-db5a-4557-a33b-e33a1d0a2c67 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:34.275212 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1137/2280] 10.222.0.15 () {54 vars in 921 bytes} [Mon Jun 27 18:47:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:47:34.281273 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf99a0b-0d4f-4376-b119-c13123c00521 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:34.281782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf99a0b-0d4f-4376-b119-c13123c00521 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:34.281974 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf99a0b-0d4f-4376-b119-c13123c00521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:34.363067 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-aaf99a0b-0d4f-4376-b119-c13123c00521 None None] MFA Rules not processed for user `1dddd4efb44c4152bb4151f7715f2223`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:34.364262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-aaf99a0b-0d4f-4376-b119-c13123c00521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:34.459049 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1144/2281] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Jun 27 18:47:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:34.472269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29e0dd-db5a-4557-a33b-e33a1d0a2c67 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:34.473976 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-6a29e0dd-db5a-4557-a33b-e33a1d0a2c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:34.487172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29e0dd-db5a-4557-a33b-e33a1d0a2c67 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:34.490654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd68333-eee3-43c1-ac36-c3a433f2ae95 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e75e5ae3ca45d0b503f6e52f399558', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:34.491688 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd68333-eee3-43c1-ac36-c3a433f2ae95 service glance] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:34.491961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd68333-eee3-43c1-ac36-c3a433f2ae95 service glance] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:34.492233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd68333-eee3-43c1-ac36-c3a433f2ae95 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:34.502107 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd68333-eee3-43c1-ac36-c3a433f2ae95 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:34.503546 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd68333-eee3-43c1-ac36-c3a433f2ae95 service glance] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:34.514877 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1138/2282] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:47:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:47:34.720192 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c8a74-b476-467a-822f-0febe0ae709a None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:34.720427 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c8a74-b476-467a-822f-0febe0ae709a None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:34.720661 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c8a74-b476-467a-822f-0febe0ae709a None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:34.722167 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1145/2283] 10.222.0.15 () {54 vars in 921 bytes} [Mon Jun 27 18:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:47:34.727025 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b818582-d895-4a10-aaa1-88cec736f079 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:34.727312 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b818582-d895-4a10-aaa1-88cec736f079 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:34.727585 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b818582-d895-4a10-aaa1-88cec736f079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:34.750100 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-8b818582-d895-4a10-aaa1-88cec736f079 None None] MFA Rules not processed for user `dbe621916dbe4f1a8686097fe2bdf762`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:34.751398 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-8b818582-d895-4a10-aaa1-88cec736f079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:34.801954 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1139/2284] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Jun 27 18:47:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:47:34.814562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c8a74-b476-467a-822f-0febe0ae709a None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:34.816305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-f00c8a74-b476-467a-822f-0febe0ae709a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:34.829824 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c8a74-b476-467a-822f-0febe0ae709a None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:34.833427 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65907d2-313a-46a1-a2f0-126ee10e5bd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:34.834053 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65907d2-313a-46a1-a2f0-126ee10e5bd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:34.834475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65907d2-313a-46a1-a2f0-126ee10e5bd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:34.834769 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65907d2-313a-46a1-a2f0-126ee10e5bd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:34.842281 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65907d2-313a-46a1-a2f0-126ee10e5bd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:34.843740 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65907d2-313a-46a1-a2f0-126ee10e5bd9 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:34.851737 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1146/2285] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:47:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:47:34.938896 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81206797-093f-44fa-bc27-99c6e4c26032 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:34.939214 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81206797-093f-44fa-bc27-99c6e4c26032 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:34.939498 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81206797-093f-44fa-bc27-99c6e4c26032 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:34.940884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1140/2286] 10.222.0.15 () {54 vars in 921 bytes} [Mon Jun 27 18:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:47:34.947268 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def29f80-d189-4e09-b437-8db13827dcab None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:34.947641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def29f80-d189-4e09-b437-8db13827dcab None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:34.947996 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def29f80-d189-4e09-b437-8db13827dcab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:34.966685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-def29f80-d189-4e09-b437-8db13827dcab None None] MFA Rules not processed for user `1dddd4efb44c4152bb4151f7715f2223`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:34.967972 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-def29f80-d189-4e09-b437-8db13827dcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:34.977611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1147/2287] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Jun 27 18:47:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:34.988167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81206797-093f-44fa-bc27-99c6e4c26032 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:34.999711 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81206797-093f-44fa-bc27-99c6e4c26032 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:35.003366 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ae719-bfee-4fde-8fa8-9143cb7007f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:35.004023 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ae719-bfee-4fde-8fa8-9143cb7007f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:35.004305 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ae719-bfee-4fde-8fa8-9143cb7007f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:35.004582 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ae719-bfee-4fde-8fa8-9143cb7007f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:35.016461 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ae719-bfee-4fde-8fa8-9143cb7007f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:35.017882 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ae719-bfee-4fde-8fa8-9143cb7007f6 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:35.028783 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1141/2288] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:47:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:47:40.385827 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe736fa7-f599-4907-aad8-d884b4cfe3ed None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:40.386327 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe736fa7-f599-4907-aad8-d884b4cfe3ed None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:40.386434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe736fa7-f599-4907-aad8-d884b4cfe3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:40.403943 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-fe736fa7-f599-4907-aad8-d884b4cfe3ed None None] MFA Rules not processed for user `58e1f1ac6ea14da0be6e9a24d007847a`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:40.405464 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-fe736fa7-f599-4907-aad8-d884b4cfe3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:40.463889 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1148/2289] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Jun 27 18:47:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:40.475977 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ae719-bfee-4fde-8fa8-9143cb7007f6 service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:40.478597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-960ae719-bfee-4fde-8fa8-9143cb7007f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:40.549653 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ae719-bfee-4fde-8fa8-9143cb7007f6 service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:40.555297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1af91-2e0f-4af9-b570-7c29b2dac78d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1529c9a45434a19968e924797688f06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:40.555935 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1af91-2e0f-4af9-b570-7c29b2dac78d service swift] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:40.556214 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1af91-2e0f-4af9-b570-7c29b2dac78d service swift] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:40.556493 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1af91-2e0f-4af9-b570-7c29b2dac78d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:40.564662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1af91-2e0f-4af9-b570-7c29b2dac78d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:40.566261 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1af91-2e0f-4af9-b570-7c29b2dac78d service swift] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:40.574438 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1142/2290] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Jun 27 18:47:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 18:47:42.319546 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe736fa7-f599-4907-aad8-d884b4cfe3ed None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:42.319546 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-fe736fa7-f599-4907-aad8-d884b4cfe3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:42.355608 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:42.425898 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe736fa7-f599-4907-aad8-d884b4cfe3ed None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:42.449017 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef6a41-e1e9-4da0-8b9f-0f69385d04ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:42.450295 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef6a41-e1e9-4da0-8b9f-0f69385d04ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:42.450295 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef6a41-e1e9-4da0-8b9f-0f69385d04ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:42.450792 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef6a41-e1e9-4da0-8b9f-0f69385d04ed admin admin] PATH_INFO: `/v3/users/618e79413e50472c80729cf2ac970109` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:42.452664 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef6a41-e1e9-4da0-8b9f-0f69385d04ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=618e79413e50472c80729cf2ac970109)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:42.454301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef6a41-e1e9-4da0-8b9f-0f69385d04ed admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:42.723827 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-78ef6a41-e1e9-4da0-8b9f-0f69385d04ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '618e79413e50472c80729cf2ac970109'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:42.752316 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-78ef6a41-e1e9-4da0-8b9f-0f69385d04ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '618e79413e50472c80729cf2ac970109'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:42.753272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-78ef6a41-e1e9-4da0-8b9f-0f69385d04ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '618e79413e50472c80729cf2ac970109'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:42.753807 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-78ef6a41-e1e9-4da0-8b9f-0f69385d04ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '618e79413e50472c80729cf2ac970109'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:42.777432 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-78ef6a41-e1e9-4da0-8b9f-0f69385d04ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '618e79413e50472c80729cf2ac970109'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:42.888736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1149/2291] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:47:42 2022] DELETE /identity/v3/users/618e79413e50472c80729cf2ac970109 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:47:42.891005 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106291f8-f51a-47bb-a0b5-0d1e5acaccdd None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:42.891519 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106291f8-f51a-47bb-a0b5-0d1e5acaccdd None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:42.891937 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106291f8-f51a-47bb-a0b5-0d1e5acaccdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:42.911210 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-106291f8-f51a-47bb-a0b5-0d1e5acaccdd None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:42.912551 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-106291f8-f51a-47bb-a0b5-0d1e5acaccdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:42.995232 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1143/2292] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:47:43.002003 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef6a41-e1e9-4da0-8b9f-0f69385d04ed admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:43.003601 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-78ef6a41-e1e9-4da0-8b9f-0f69385d04ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:43.069096 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef6a41-e1e9-4da0-8b9f-0f69385d04ed admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:43.073920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b6a762-232c-4acb-a763-d73f472b104f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:43.074729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6a762-232c-4acb-a763-d73f472b104f service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:43.075083 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6a762-232c-4acb-a763-d73f472b104f service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:43.075364 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b6a762-232c-4acb-a763-d73f472b104f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:43.086241 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b6a762-232c-4acb-a763-d73f472b104f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:43.087732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b6a762-232c-4acb-a763-d73f472b104f service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:43.097830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1150/2293] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:47:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:47:43.749515 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106291f8-f51a-47bb-a0b5-0d1e5acaccdd None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:43.751364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-106291f8-f51a-47bb-a0b5-0d1e5acaccdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:43.764719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106291f8-f51a-47bb-a0b5-0d1e5acaccdd None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:43.769824 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bcf4c9-d57a-4202-a7af-518193475104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:43.770734 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcf4c9-d57a-4202-a7af-518193475104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:43.771019 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcf4c9-d57a-4202-a7af-518193475104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:43.771294 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcf4c9-d57a-4202-a7af-518193475104 admin admin] PATH_INFO: `/v3/users/aa78b9c2e6f74103b61de91798b43b25` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:43.772528 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bcf4c9-d57a-4202-a7af-518193475104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa78b9c2e6f74103b61de91798b43b25)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:43.774649 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bcf4c9-d57a-4202-a7af-518193475104 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:43.781266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:44.000257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b6a762-232c-4acb-a763-d73f472b104f service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:44.001752 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-08b6a762-232c-4acb-a763-d73f472b104f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:44.019054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b6a762-232c-4acb-a763-d73f472b104f service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:44.026057 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b09ac9-751d-4773-8248-e31b66fd1aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:44.027138 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b09ac9-751d-4773-8248-e31b66fd1aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:44.027422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b09ac9-751d-4773-8248-e31b66fd1aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:44.027694 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b09ac9-751d-4773-8248-e31b66fd1aa8 admin admin] PATH_INFO: `/v3/projects/46727708a5de48f08e1e6b9cc4b7a82b` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:44.029932 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b09ac9-751d-4773-8248-e31b66fd1aa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46727708a5de48f08e1e6b9cc4b7a82b)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:44.034944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b09ac9-751d-4773-8248-e31b66fd1aa8 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:44.459960 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:44.467943 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-a0bcf4c9-d57a-4202-a7af-518193475104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa78b9c2e6f74103b61de91798b43b25'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:44.471968 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f6b09ac9-751d-4773-8248-e31b66fd1aa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46727708a5de48f08e1e6b9cc4b7a82b', 'request_id': 'req-f6b09ac9-751d-4773-8248-e31b66fd1aa8'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:44.477282 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-a0bcf4c9-d57a-4202-a7af-518193475104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa78b9c2e6f74103b61de91798b43b25'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:44.478000 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-a0bcf4c9-d57a-4202-a7af-518193475104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa78b9c2e6f74103b61de91798b43b25'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:44.492136 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-a0bcf4c9-d57a-4202-a7af-518193475104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa78b9c2e6f74103b61de91798b43b25'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:44.544817 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f6b09ac9-751d-4773-8248-e31b66fd1aa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46727708a5de48f08e1e6b9cc4b7a82b', 'request_id': 'req-f6b09ac9-751d-4773-8248-e31b66fd1aa8'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:44.552708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f6b09ac9-751d-4773-8248-e31b66fd1aa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46727708a5de48f08e1e6b9cc4b7a82b', 'request_id': 'req-f6b09ac9-751d-4773-8248-e31b66fd1aa8'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:44.565477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f6b09ac9-751d-4773-8248-e31b66fd1aa8 admin admin] The token cache is being invalidate because project 46727708a5de48f08e1e6b9cc4b7a82b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:47:44.565895 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f6b09ac9-751d-4773-8248-e31b66fd1aa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:44.568131 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f6b09ac9-751d-4773-8248-e31b66fd1aa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:44.574640 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1151/2294] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:47:43 2022] DELETE /identity/v3/projects/46727708a5de48f08e1e6b9cc4b7a82b => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:47:44.576214 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c70ff2-c6ae-451d-ba68-195a5d970ffb None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:44.576812 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c70ff2-c6ae-451d-ba68-195a5d970ffb None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:44.577162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c70ff2-c6ae-451d-ba68-195a5d970ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:44.598071 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-f0c70ff2-c6ae-451d-ba68-195a5d970ffb None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:44.599667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-f0c70ff2-c6ae-451d-ba68-195a5d970ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:44.618818 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-a0bcf4c9-d57a-4202-a7af-518193475104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa78b9c2e6f74103b61de91798b43b25'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:44.626306 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1144/2295] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:47:43 2022] DELETE /identity/v3/users/aa78b9c2e6f74103b61de91798b43b25 => generated 0 bytes in 877 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:47:44.644342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf371c-c553-413a-8ada-bb73b90b090e None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:44.644342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf371c-c553-413a-8ada-bb73b90b090e None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:44.644342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf371c-c553-413a-8ada-bb73b90b090e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:44.653401 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-73bf371c-c553-413a-8ada-bb73b90b090e None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:44.655644 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-73bf371c-c553-413a-8ada-bb73b90b090e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:44.678498 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1152/2296] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:44.685809 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef46e00-8c8d-42c8-b4b4-a3c121f81e94 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:44.686101 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef46e00-8c8d-42c8-b4b4-a3c121f81e94 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:44.686410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef46e00-8c8d-42c8-b4b4-a3c121f81e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:44.705968 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-7ef46e00-8c8d-42c8-b4b4-a3c121f81e94 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:44.707556 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-7ef46e00-8c8d-42c8-b4b4-a3c121f81e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:44.745374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1145/2297] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:47:44.756291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf371c-c553-413a-8ada-bb73b90b090e None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:44.759100 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-73bf371c-c553-413a-8ada-bb73b90b090e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:44.819602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1153/2298] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:44.828195 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef46e00-8c8d-42c8-b4b4-a3c121f81e94 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:44.848626 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef46e00-8c8d-42c8-b4b4-a3c121f81e94 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:44.854869 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf371c-c553-413a-8ada-bb73b90b090e None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:44.868716 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbf258-4dda-4302-9fb1-f82eae870440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:44.872745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbf258-4dda-4302-9fb1-f82eae870440 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:44.872745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbf258-4dda-4302-9fb1-f82eae870440 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:44.872745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbf258-4dda-4302-9fb1-f82eae870440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:44.873037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba5440-da6a-4f6d-add1-c029e5b35b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:44.873141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba5440-da6a-4f6d-add1-c029e5b35b42 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:44.873545 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba5440-da6a-4f6d-add1-c029e5b35b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:44.873939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba5440-da6a-4f6d-add1-c029e5b35b42 admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:44.875401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba5440-da6a-4f6d-add1-c029e5b35b42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:47:44.875775 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba5440-da6a-4f6d-add1-c029e5b35b42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:44.877757 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba5440-da6a-4f6d-add1-c029e5b35b42 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:44.894748 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dbf258-4dda-4302-9fb1-f82eae870440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:44.897210 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dbf258-4dda-4302-9fb1-f82eae870440 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:44.906275 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1154/2299] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:47:44.907536 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba5440-da6a-4f6d-add1-c029e5b35b42 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:44.915674 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1146/2300] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:47:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:47:44.919283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba5440-da6a-4f6d-add1-c029e5b35b42 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:44.925643 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378837c-a7f7-41b0-ae5a-52f561a13876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:44.926762 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378837c-a7f7-41b0-ae5a-52f561a13876 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:44.927286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378837c-a7f7-41b0-ae5a-52f561a13876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:44.927572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378837c-a7f7-41b0-ae5a-52f561a13876 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:44.928297 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378837c-a7f7-41b0-ae5a-52f561a13876 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:44.930561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378837c-a7f7-41b0-ae5a-52f561a13876 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:45.059537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1155/2301] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:47:44 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:47:45.064349 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbf258-4dda-4302-9fb1-f82eae870440 service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:45.075927 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbf258-4dda-4302-9fb1-f82eae870440 service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:45.157808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:45.221644 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5474c6-b16d-44f3-952a-7249fcca2eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:45.222339 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5474c6-b16d-44f3-952a-7249fcca2eb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:45.222656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5474c6-b16d-44f3-952a-7249fcca2eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:45.223104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5474c6-b16d-44f3-952a-7249fcca2eb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:45.223832 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5474c6-b16d-44f3-952a-7249fcca2eb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:45.229029 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5474c6-b16d-44f3-952a-7249fcca2eb6 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:45.338798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378837c-a7f7-41b0-ae5a-52f561a13876 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:45.339929 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1147/2302] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:47:45 2022] POST /identity/v3/users => generated 292 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:47:45.359846 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378837c-a7f7-41b0-ae5a-52f561a13876 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:45.369342 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd385652-dd23-403a-9240-ef9eb017b34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:45.370021 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd385652-dd23-403a-9240-ef9eb017b34f admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:45.370308 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd385652-dd23-403a-9240-ef9eb017b34f admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:45.370613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd385652-dd23-403a-9240-ef9eb017b34f admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:45.371352 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd385652-dd23-403a-9240-ef9eb017b34f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:45.372959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd385652-dd23-403a-9240-ef9eb017b34f admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:45.400958 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5474c6-b16d-44f3-952a-7249fcca2eb6 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:45.402176 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1156/2303] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:47:45.414339 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5474c6-b16d-44f3-952a-7249fcca2eb6 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:45.420295 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da456c98-8ec8-4a53-a5d4-10db1b5fe9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:45.421151 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da456c98-8ec8-4a53-a5d4-10db1b5fe9f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:45.421438 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da456c98-8ec8-4a53-a5d4-10db1b5fe9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:45.421797 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da456c98-8ec8-4a53-a5d4-10db1b5fe9f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:45.422721 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da456c98-8ec8-4a53-a5d4-10db1b5fe9f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:45.432472 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da456c98-8ec8-4a53-a5d4-10db1b5fe9f3 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:45.440636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1148/2304] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:47:45.441743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd385652-dd23-403a-9240-ef9eb017b34f admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:45.454161 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd385652-dd23-403a-9240-ef9eb017b34f admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:45.459656 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589b1613-faaa-4043-bcbe-31ad25f7345c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:45.461733 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b1613-faaa-4043-bcbe-31ad25f7345c admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:45.462029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b1613-faaa-4043-bcbe-31ad25f7345c admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:45.462314 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b1613-faaa-4043-bcbe-31ad25f7345c admin admin] PATH_INFO: `/v3/projects/5284ec5970e14418bfbd5391ee3b282c/users/f5f17909688f40f9b752e63b4074c2ee/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:45.482618 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589b1613-faaa-4043-bcbe-31ad25f7345c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5284ec5970e14418bfbd5391ee3b282c, user_id=f5f17909688f40f9b752e63b4074c2ee, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:45.485305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589b1613-faaa-4043-bcbe-31ad25f7345c admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:45.489265 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:45.532851 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da456c98-8ec8-4a53-a5d4-10db1b5fe9f3 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:45.534927 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-da456c98-8ec8-4a53-a5d4-10db1b5fe9f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:45.556204 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da456c98-8ec8-4a53-a5d4-10db1b5fe9f3 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:45.578430 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670703ee-516d-43a2-9d72-aeccad957cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:45.582506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670703ee-516d-43a2-9d72-aeccad957cbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:45.587730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670703ee-516d-43a2-9d72-aeccad957cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:45.588255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670703ee-516d-43a2-9d72-aeccad957cbe admin admin] PATH_INFO: `/v3/projects/0f87955d76bd4456b0b696a18b89b902` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:45.590734 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670703ee-516d-43a2-9d72-aeccad957cbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f87955d76bd4456b0b696a18b89b902)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:45.591063 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1157/2305] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:47:45 2022] PUT /identity/v3/projects/5284ec5970e14418bfbd5391ee3b282c/users/f5f17909688f40f9b752e63b4074c2ee/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:47:45.593080 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670703ee-516d-43a2-9d72-aeccad957cbe admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:45.596648 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33811fa-72d9-4f18-9f09-3fcddc69c594 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:45.596971 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33811fa-72d9-4f18-9f09-3fcddc69c594 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:45.597280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33811fa-72d9-4f18-9f09-3fcddc69c594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:45.654468 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-b33811fa-72d9-4f18-9f09-3fcddc69c594 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:45.655992 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-b33811fa-72d9-4f18-9f09-3fcddc69c594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:45.790004 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1158/2306] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:45.793834 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-670703ee-516d-43a2-9d72-aeccad957cbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f87955d76bd4456b0b696a18b89b902', 'request_id': 'req-670703ee-516d-43a2-9d72-aeccad957cbe'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:45.808332 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33811fa-72d9-4f18-9f09-3fcddc69c594 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:45.863474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-670703ee-516d-43a2-9d72-aeccad957cbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f87955d76bd4456b0b696a18b89b902', 'request_id': 'req-670703ee-516d-43a2-9d72-aeccad957cbe'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:45.878511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-670703ee-516d-43a2-9d72-aeccad957cbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f87955d76bd4456b0b696a18b89b902', 'request_id': 'req-670703ee-516d-43a2-9d72-aeccad957cbe'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:45.901437 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-670703ee-516d-43a2-9d72-aeccad957cbe admin admin] The token cache is being invalidate because project 0f87955d76bd4456b0b696a18b89b902 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:47:45.901749 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-670703ee-516d-43a2-9d72-aeccad957cbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:45.902462 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-670703ee-516d-43a2-9d72-aeccad957cbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:45.904530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1149/2307] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:47:45 2022] DELETE /identity/v3/projects/0f87955d76bd4456b0b696a18b89b902 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:47:45.909242 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c53ab-1b47-496c-9636-dd78c2185c91 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:45.909862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c53ab-1b47-496c-9636-dd78c2185c91 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:45.910326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c53ab-1b47-496c-9636-dd78c2185c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:45.921163 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33811fa-72d9-4f18-9f09-3fcddc69c594 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:45.942875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-e66c53ab-1b47-496c-9636-dd78c2185c91 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:45.944098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-e66c53ab-1b47-496c-9636-dd78c2185c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:45.988796 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9000e0-b991-40f2-816f-8fd4d2e11a98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:45.989729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9000e0-b991-40f2-816f-8fd4d2e11a98 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:45.990124 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9000e0-b991-40f2-816f-8fd4d2e11a98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:45.990463 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9000e0-b991-40f2-816f-8fd4d2e11a98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:45.993056 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-5d9000e0-b991-40f2-816f-8fd4d2e11a98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:46.037762 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1150/2308] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:47:46.046212 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce330239-c348-4d32-9361-cf3c8bd0409e None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:46.049962 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce330239-c348-4d32-9361-cf3c8bd0409e None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:46.049962 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce330239-c348-4d32-9361-cf3c8bd0409e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:46.079432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-ce330239-c348-4d32-9361-cf3c8bd0409e None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:46.081995 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-ce330239-c348-4d32-9361-cf3c8bd0409e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:46.095694 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1151/2309] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:46.099657 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce330239-c348-4d32-9361-cf3c8bd0409e None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:46.115187 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce330239-c348-4d32-9361-cf3c8bd0409e None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:46.122121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9000e0-b991-40f2-816f-8fd4d2e11a98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:46.124336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41192ba4-1c08-45c0-b1e1-f0c24b5f2f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:46.124782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9000e0-b991-40f2-816f-8fd4d2e11a98 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:46.125383 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41192ba4-1c08-45c0-b1e1-f0c24b5f2f4f admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:46.125727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41192ba4-1c08-45c0-b1e1-f0c24b5f2f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:46.126091 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41192ba4-1c08-45c0-b1e1-f0c24b5f2f4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:46.126934 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41192ba4-1c08-45c0-b1e1-f0c24b5f2f4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:47:46.130522 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41192ba4-1c08-45c0-b1e1-f0c24b5f2f4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:46.130522 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41192ba4-1c08-45c0-b1e1-f0c24b5f2f4f admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:46.140137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1159/2310] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:47:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:47:46.153677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1152/2311] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:47:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:47:46.154959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9000e0-b991-40f2-816f-8fd4d2e11a98 service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:46.161271 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:46.168930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9000e0-b991-40f2-816f-8fd4d2e11a98 service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:46.176354 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8ab1c-842e-436e-9af7-79fd013b3d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:46.177041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8ab1c-842e-436e-9af7-79fd013b3d39 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:46.177390 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8ab1c-842e-436e-9af7-79fd013b3d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:46.177734 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8ab1c-842e-436e-9af7-79fd013b3d39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:46.178534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8ab1c-842e-436e-9af7-79fd013b3d39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:46.181262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8ab1c-842e-436e-9af7-79fd013b3d39 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:46.286647 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1160/2312] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:47:46 2022] POST /identity/v3/projects => generated 371 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:47:46.291296 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41192ba4-1c08-45c0-b1e1-f0c24b5f2f4f admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:46.303743 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41192ba4-1c08-45c0-b1e1-f0c24b5f2f4f admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:46.332898 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:46.420900 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bce4a0-83ec-48a4-ad04-8b129d6c25f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:46.422199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bce4a0-83ec-48a4-ad04-8b129d6c25f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:46.422670 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bce4a0-83ec-48a4-ad04-8b129d6c25f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:46.423333 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bce4a0-83ec-48a4-ad04-8b129d6c25f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:46.424711 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bce4a0-83ec-48a4-ad04-8b129d6c25f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:46.428083 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bce4a0-83ec-48a4-ad04-8b129d6c25f0 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:46.522194 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1153/2313] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:47:46 2022] POST /identity/v3/users => generated 286 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:47:46.527010 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8ab1c-842e-436e-9af7-79fd013b3d39 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:46.542028 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8ab1c-842e-436e-9af7-79fd013b3d39 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:46.551169 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f476b2-daa3-4e47-b695-b5ffdc3d364a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:46.551932 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f476b2-daa3-4e47-b695-b5ffdc3d364a admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:46.552222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f476b2-daa3-4e47-b695-b5ffdc3d364a admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:46.552937 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f476b2-daa3-4e47-b695-b5ffdc3d364a admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:46.558097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f476b2-daa3-4e47-b695-b5ffdc3d364a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:46.563773 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f476b2-daa3-4e47-b695-b5ffdc3d364a admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:46.580132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1161/2314] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:47:46 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:47:46.584322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bce4a0-83ec-48a4-ad04-8b129d6c25f0 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:46.599717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bce4a0-83ec-48a4-ad04-8b129d6c25f0 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:46.608401 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97568ca6-4cfc-4ec6-b358-86a4a8b11de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:46.609635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97568ca6-4cfc-4ec6-b358-86a4a8b11de5 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:46.610170 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97568ca6-4cfc-4ec6-b358-86a4a8b11de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:46.610543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97568ca6-4cfc-4ec6-b358-86a4a8b11de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:46.612217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97568ca6-4cfc-4ec6-b358-86a4a8b11de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:46.615627 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97568ca6-4cfc-4ec6-b358-86a4a8b11de5 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:46.634788 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1154/2315] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:47:46 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:47:46.638045 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f476b2-daa3-4e47-b695-b5ffdc3d364a admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:46.649166 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f476b2-daa3-4e47-b695-b5ffdc3d364a admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:46.654812 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da12f7-677f-4422-856f-df4fd79b58d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:46.655977 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da12f7-677f-4422-856f-df4fd79b58d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:46.663002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da12f7-677f-4422-856f-df4fd79b58d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:46.663002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da12f7-677f-4422-856f-df4fd79b58d6 admin admin] PATH_INFO: `/v3/projects/2e8fc4b97c74407bbf235ef08f8797d2/users/63441402d47444d98d9ac91488a08aa1/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:46.680124 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da12f7-677f-4422-856f-df4fd79b58d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e8fc4b97c74407bbf235ef08f8797d2, user_id=63441402d47444d98d9ac91488a08aa1, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:46.683356 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da12f7-677f-4422-856f-df4fd79b58d6 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:46.752937 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1162/2316] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:47:46 2022] PUT /identity/v3/projects/2e8fc4b97c74407bbf235ef08f8797d2/users/63441402d47444d98d9ac91488a08aa1/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:47:46.768020 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bd434-888c-436a-8821-c2976a593ef7 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:46.768609 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bd434-888c-436a-8821-c2976a593ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:46.769104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bd434-888c-436a-8821-c2976a593ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:46.813905 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-443bd434-888c-436a-8821-c2976a593ef7 None None] MFA Rules not processed for user `63441402d47444d98d9ac91488a08aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:46.816001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-443bd434-888c-436a-8821-c2976a593ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:46.931178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da12f7-677f-4422-856f-df4fd79b58d6 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:46.934611 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-64da12f7-677f-4422-856f-df4fd79b58d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:46.996933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1155/2317] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:47.011290 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bd434-888c-436a-8821-c2976a593ef7 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:47.012885 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-443bd434-888c-436a-8821-c2976a593ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:47.109630 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da12f7-677f-4422-856f-df4fd79b58d6 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:47.127181 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1ebab-154b-4554-ab51-2022c95e7858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:47.128071 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1ebab-154b-4554-ab51-2022c95e7858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:47.128521 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1ebab-154b-4554-ab51-2022c95e7858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:47.129498 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1ebab-154b-4554-ab51-2022c95e7858 admin admin] PATH_INFO: `/v3/users/745099adc24a4b12b3e22cfadd29bd70` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:47.130840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1ebab-154b-4554-ab51-2022c95e7858 admin admin] RBAC: Authorizing `identity:delete_user(user_id=745099adc24a4b12b3e22cfadd29bd70)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:47.133598 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1ebab-154b-4554-ab51-2022c95e7858 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:47.137006 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bd434-888c-436a-8821-c2976a593ef7 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:47.140990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1638a-7518-4f8e-af98-b32b17319cd6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e75e5ae3ca45d0b503f6e52f399558', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:47.141767 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1638a-7518-4f8e-af98-b32b17319cd6 service glance] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:47.142155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1638a-7518-4f8e-af98-b32b17319cd6 service glance] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:47.142587 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1638a-7518-4f8e-af98-b32b17319cd6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:47.152739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1638a-7518-4f8e-af98-b32b17319cd6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:47.154802 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1638a-7518-4f8e-af98-b32b17319cd6 service glance] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:47.169163 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1156/2318] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:47:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:47:47.391620 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-e9f1ebab-154b-4554-ab51-2022c95e7858 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '745099adc24a4b12b3e22cfadd29bd70'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:47.412476 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-e9f1ebab-154b-4554-ab51-2022c95e7858 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '745099adc24a4b12b3e22cfadd29bd70'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:47.416483 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-e9f1ebab-154b-4554-ab51-2022c95e7858 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '745099adc24a4b12b3e22cfadd29bd70'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:47.418489 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-e9f1ebab-154b-4554-ab51-2022c95e7858 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '745099adc24a4b12b3e22cfadd29bd70'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:47.450925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-e9f1ebab-154b-4554-ab51-2022c95e7858 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '745099adc24a4b12b3e22cfadd29bd70'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:47.557444 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1163/2319] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:47:46 2022] DELETE /identity/v3/users/745099adc24a4b12b3e22cfadd29bd70 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:47:47.562578 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a86098-2610-4411-b83e-681c8fe187fd None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:47.562872 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a86098-2610-4411-b83e-681c8fe187fd None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:47.563155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a86098-2610-4411-b83e-681c8fe187fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:47.598109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-68a86098-2610-4411-b83e-681c8fe187fd None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:47.600328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-68a86098-2610-4411-b83e-681c8fe187fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:47.639969 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:47.754686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1157/2320] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:47.783516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1ebab-154b-4554-ab51-2022c95e7858 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:47.790551 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-e9f1ebab-154b-4554-ab51-2022c95e7858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:47.840094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:47.928784 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1ebab-154b-4554-ab51-2022c95e7858 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:47.933786 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47352395-da3b-4f45-9868-19f94d124961 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:47.934476 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47352395-da3b-4f45-9868-19f94d124961 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:47.934788 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47352395-da3b-4f45-9868-19f94d124961 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:47.935023 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47352395-da3b-4f45-9868-19f94d124961 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:47.949876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47352395-da3b-4f45-9868-19f94d124961 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:47.951566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47352395-da3b-4f45-9868-19f94d124961 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:47.966202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1164/2321] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:47:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:47:48.425443 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f338f-8f31-4e9a-a612-d191d331c44a None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:48.425799 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f338f-8f31-4e9a-a612-d191d331c44a None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:48.426161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f338f-8f31-4e9a-a612-d191d331c44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:48.489037 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-f33f338f-8f31-4e9a-a612-d191d331c44a None None] MFA Rules not processed for user `f5f17909688f40f9b752e63b4074c2ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:48.490268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f33f338f-8f31-4e9a-a612-d191d331c44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:48.623682 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1158/2322] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:48.635956 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47352395-da3b-4f45-9868-19f94d124961 service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:48.637478 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-47352395-da3b-4f45-9868-19f94d124961 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:48.654129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f338f-8f31-4e9a-a612-d191d331c44a None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:48.660408 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f33f338f-8f31-4e9a-a612-d191d331c44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:48.699850 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f338f-8f31-4e9a-a612-d191d331c44a None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:48.713959 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1319dda-4d03-4a23-b472-f5ca575f056c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:48.713959 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1319dda-4d03-4a23-b472-f5ca575f056c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:48.713959 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1319dda-4d03-4a23-b472-f5ca575f056c admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:48.713959 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1319dda-4d03-4a23-b472-f5ca575f056c admin admin] PATH_INFO: `/v3/projects/dafa445887284fe7a018c479dd8335cb` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:48.714334 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1319dda-4d03-4a23-b472-f5ca575f056c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dafa445887284fe7a018c479dd8335cb)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:48.716623 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1319dda-4d03-4a23-b472-f5ca575f056c admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:48.853029 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f1319dda-4d03-4a23-b472-f5ca575f056c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dafa445887284fe7a018c479dd8335cb', 'request_id': 'req-f1319dda-4d03-4a23-b472-f5ca575f056c'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:48.854342 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47352395-da3b-4f45-9868-19f94d124961 service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:48.916300 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f1d2c-489f-4882-81dc-b15f63705c04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:48.917097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f1d2c-489f-4882-81dc-b15f63705c04 service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:48.917528 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f1d2c-489f-4882-81dc-b15f63705c04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:48.917953 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f1d2c-489f-4882-81dc-b15f63705c04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:48.931039 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f1319dda-4d03-4a23-b472-f5ca575f056c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dafa445887284fe7a018c479dd8335cb', 'request_id': 'req-f1319dda-4d03-4a23-b472-f5ca575f056c'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:48.938298 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f1319dda-4d03-4a23-b472-f5ca575f056c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dafa445887284fe7a018c479dd8335cb', 'request_id': 'req-f1319dda-4d03-4a23-b472-f5ca575f056c'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:48.950476 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f1319dda-4d03-4a23-b472-f5ca575f056c admin admin] The token cache is being invalidate because project dafa445887284fe7a018c479dd8335cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:47:48.950826 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f1319dda-4d03-4a23-b472-f5ca575f056c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:48.951629 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f1319dda-4d03-4a23-b472-f5ca575f056c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:47:48.956765 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1159/2323] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:47:48 2022] DELETE /identity/v3/projects/dafa445887284fe7a018c479dd8335cb => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:47:48.958346 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a4ca1-4462-4ddf-a469-8ff4a0d2f872 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:48.958756 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a4ca1-4462-4ddf-a469-8ff4a0d2f872 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:48.959071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a4ca1-4462-4ddf-a469-8ff4a0d2f872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:48.983921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-939a4ca1-4462-4ddf-a469-8ff4a0d2f872 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:48.985635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-939a4ca1-4462-4ddf-a469-8ff4a0d2f872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:49.034287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:49.037954 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f1d2c-489f-4882-81dc-b15f63705c04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:49.040138 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f1d2c-489f-4882-81dc-b15f63705c04 service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:49.052390 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1165/2324] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:47:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:47:49.089239 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1160/2325] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:49.097951 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16f128-5e7d-4bf2-863f-2a11262be13e None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:49.098239 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16f128-5e7d-4bf2-863f-2a11262be13e None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:49.098563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16f128-5e7d-4bf2-863f-2a11262be13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:49.142329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-ca16f128-5e7d-4bf2-863f-2a11262be13e None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:49.144552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ca16f128-5e7d-4bf2-863f-2a11262be13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:49.172281 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1166/2326] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:47:49.177498 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a4ca1-4462-4ddf-a469-8ff4a0d2f872 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:49.202823 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a4ca1-4462-4ddf-a469-8ff4a0d2f872 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:49.210598 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb341e0-5d66-4361-9d89-2771b8da95e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:49.211339 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb341e0-5d66-4361-9d89-2771b8da95e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:49.211625 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb341e0-5d66-4361-9d89-2771b8da95e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:49.211902 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb341e0-5d66-4361-9d89-2771b8da95e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:49.213093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb341e0-5d66-4361-9d89-2771b8da95e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:47:49.213782 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb341e0-5d66-4361-9d89-2771b8da95e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:49.227179 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb341e0-5d66-4361-9d89-2771b8da95e5 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:49.255818 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1161/2327] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:47:49.260934 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16f128-5e7d-4bf2-863f-2a11262be13e None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:49.273189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16f128-5e7d-4bf2-863f-2a11262be13e None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:49.282997 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649e6f4-8adb-479d-a01f-ad9acb577b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:49.284257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649e6f4-8adb-479d-a01f-ad9acb577b80 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:49.284817 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649e6f4-8adb-479d-a01f-ad9acb577b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:49.285474 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649e6f4-8adb-479d-a01f-ad9acb577b80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:49.286617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a649e6f4-8adb-479d-a01f-ad9acb577b80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:49.290200 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a649e6f4-8adb-479d-a01f-ad9acb577b80 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:49.302080 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:49.436696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1167/2328] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:47:49 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:47:49.438472 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb341e0-5d66-4361-9d89-2771b8da95e5 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:49.453325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb341e0-5d66-4361-9d89-2771b8da95e5 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:49.540791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a193b27-9df2-45b3-a84f-d16d27209810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:49.541591 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a193b27-9df2-45b3-a84f-d16d27209810 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:49.541976 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a193b27-9df2-45b3-a84f-d16d27209810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:49.542707 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a193b27-9df2-45b3-a84f-d16d27209810 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:49.543526 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a193b27-9df2-45b3-a84f-d16d27209810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:49.545964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a193b27-9df2-45b3-a84f-d16d27209810 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:49.642909 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1162/2329] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:47:49 2022] POST /identity/v3/users => generated 289 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:47:49.645948 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649e6f4-8adb-479d-a01f-ad9acb577b80 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:49.659619 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649e6f4-8adb-479d-a01f-ad9acb577b80 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:49.677616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6455f-5beb-47f0-96f1-e1a4374bb38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:49.677616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6455f-5beb-47f0-96f1-e1a4374bb38d admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:49.677616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6455f-5beb-47f0-96f1-e1a4374bb38d admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:49.677616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6455f-5beb-47f0-96f1-e1a4374bb38d admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:49.678026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca6455f-5beb-47f0-96f1-e1a4374bb38d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:49.678026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca6455f-5beb-47f0-96f1-e1a4374bb38d admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:49.690303 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1168/2330] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:47:49.697003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a193b27-9df2-45b3-a84f-d16d27209810 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:49.709652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a193b27-9df2-45b3-a84f-d16d27209810 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:49.714576 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575266b7-4246-4a53-971c-8dde6d3c9f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:49.715236 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575266b7-4246-4a53-971c-8dde6d3c9f54 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:49.715518 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575266b7-4246-4a53-971c-8dde6d3c9f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:49.715791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575266b7-4246-4a53-971c-8dde6d3c9f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:49.716514 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575266b7-4246-4a53-971c-8dde6d3c9f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:49.718198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575266b7-4246-4a53-971c-8dde6d3c9f54 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:49.729122 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1163/2331] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:47:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:47:49.733702 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6455f-5beb-47f0-96f1-e1a4374bb38d admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:49.747880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6455f-5beb-47f0-96f1-e1a4374bb38d admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:49.758560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638b3ff-1f4c-4ebd-a6f2-c639583a8a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:49.759497 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638b3ff-1f4c-4ebd-a6f2-c639583a8a40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:49.759778 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638b3ff-1f4c-4ebd-a6f2-c639583a8a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:49.760051 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638b3ff-1f4c-4ebd-a6f2-c639583a8a40 admin admin] PATH_INFO: `/v3/projects/d7ecbf4573cc4b57b7564fa10664d7be/users/7dbe40e2b8d548d4a6226e171f016495/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:49.775990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2638b3ff-1f4c-4ebd-a6f2-c639583a8a40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7ecbf4573cc4b57b7564fa10664d7be, user_id=7dbe40e2b8d548d4a6226e171f016495, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:49.778727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2638b3ff-1f4c-4ebd-a6f2-c639583a8a40 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:49.863121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1169/2332] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:47:49 2022] PUT /identity/v3/projects/d7ecbf4573cc4b57b7564fa10664d7be/users/7dbe40e2b8d548d4a6226e171f016495/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:47:49.874943 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0fbf6-53e8-46fb-be2b-63d51004d17c None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:49.875261 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0fbf6-53e8-46fb-be2b-63d51004d17c None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:49.875544 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0fbf6-53e8-46fb-be2b-63d51004d17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:49.924530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-54c0fbf6-53e8-46fb-be2b-63d51004d17c None None] MFA Rules not processed for user `7dbe40e2b8d548d4a6226e171f016495`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:49.931937 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-54c0fbf6-53e8-46fb-be2b-63d51004d17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:50.020328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1164/2333] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:50.025926 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638b3ff-1f4c-4ebd-a6f2-c639583a8a40 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:50.098550 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638b3ff-1f4c-4ebd-a6f2-c639583a8a40 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:50.104105 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a3706-2546-4033-bcde-c5fbb2a4d2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:50.105018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a3706-2546-4033-bcde-c5fbb2a4d2b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:50.105304 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a3706-2546-4033-bcde-c5fbb2a4d2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:50.105581 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a3706-2546-4033-bcde-c5fbb2a4d2b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:50.106332 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a3706-2546-4033-bcde-c5fbb2a4d2b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:50.108989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a3706-2546-4033-bcde-c5fbb2a4d2b8 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:50.213133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1170/2334] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:47:50 2022] POST /identity/v3/projects => generated 375 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:47:50.217262 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0fbf6-53e8-46fb-be2b-63d51004d17c None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:50.229056 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0fbf6-53e8-46fb-be2b-63d51004d17c None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:50.261804 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:50.290658 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3c24a-b599-4d88-a0dc-7f3d617ff048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:50.291366 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3c24a-b599-4d88-a0dc-7f3d617ff048 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:50.291652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3c24a-b599-4d88-a0dc-7f3d617ff048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:50.291924 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3c24a-b599-4d88-a0dc-7f3d617ff048 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:50.292622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3c24a-b599-4d88-a0dc-7f3d617ff048 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:50.294575 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3c24a-b599-4d88-a0dc-7f3d617ff048 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:50.393235 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1165/2335] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:47:50 2022] POST /identity/v3/users => generated 288 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:47:50.394474 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a3706-2546-4033-bcde-c5fbb2a4d2b8 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:50.406898 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a3706-2546-4033-bcde-c5fbb2a4d2b8 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:50.412940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a6d8a-a295-4992-937c-1c3e87f7784f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:50.414330 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a6d8a-a295-4992-937c-1c3e87f7784f admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:50.414724 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a6d8a-a295-4992-937c-1c3e87f7784f admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:50.415005 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a6d8a-a295-4992-937c-1c3e87f7784f admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:50.415740 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a6d8a-a295-4992-937c-1c3e87f7784f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:50.417352 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a6d8a-a295-4992-937c-1c3e87f7784f admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:50.428533 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1171/2336] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:47:50.432934 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3c24a-b599-4d88-a0dc-7f3d617ff048 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:50.443589 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3c24a-b599-4d88-a0dc-7f3d617ff048 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:50.449304 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173d21d-7f44-42b1-acd6-78bbd923947a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:50.452208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173d21d-7f44-42b1-acd6-78bbd923947a admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:50.452208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173d21d-7f44-42b1-acd6-78bbd923947a admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:50.452208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173d21d-7f44-42b1-acd6-78bbd923947a admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:50.452208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173d21d-7f44-42b1-acd6-78bbd923947a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:50.456981 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173d21d-7f44-42b1-acd6-78bbd923947a admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:50.473401 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1166/2337] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:47:50.474671 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a6d8a-a295-4992-937c-1c3e87f7784f admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:50.487625 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a6d8a-a295-4992-937c-1c3e87f7784f admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:50.493836 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b635d-af6a-4cec-a559-75c57b9966dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:50.494898 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b635d-af6a-4cec-a559-75c57b9966dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:50.495202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b635d-af6a-4cec-a559-75c57b9966dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:50.495462 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b635d-af6a-4cec-a559-75c57b9966dc admin admin] PATH_INFO: `/v3/projects/d5e39794c15d4c5ea19569dc4d6de4c9/users/d3e12bc3c2c4478c89b17bd249befd0d/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:50.514204 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b635d-af6a-4cec-a559-75c57b9966dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e39794c15d4c5ea19569dc4d6de4c9, user_id=d3e12bc3c2c4478c89b17bd249befd0d, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:50.517181 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b635d-af6a-4cec-a559-75c57b9966dc admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:50.595754 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1172/2338] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:47:50 2022] PUT /identity/v3/projects/d5e39794c15d4c5ea19569dc4d6de4c9/users/d3e12bc3c2c4478c89b17bd249befd0d/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:47:50.608663 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49c987-54b5-43c4-8d8e-3d82669c5e42 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:50.608947 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49c987-54b5-43c4-8d8e-3d82669c5e42 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:50.609431 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49c987-54b5-43c4-8d8e-3d82669c5e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:50.642882 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-0b49c987-54b5-43c4-8d8e-3d82669c5e42 None None] MFA Rules not processed for user `d3e12bc3c2c4478c89b17bd249befd0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:50.644133 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-0b49c987-54b5-43c4-8d8e-3d82669c5e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:50.718144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1167/2339] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:50.722545 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b635d-af6a-4cec-a559-75c57b9966dc admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:50.800392 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b635d-af6a-4cec-a559-75c57b9966dc admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:50.807550 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb3ee5-c274-4610-abcd-e3bf697075ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:50.808263 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb3ee5-c274-4610-abcd-e3bf697075ad admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:50.808549 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb3ee5-c274-4610-abcd-e3bf697075ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:50.808836 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb3ee5-c274-4610-abcd-e3bf697075ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:50.809566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb3ee5-c274-4610-abcd-e3bf697075ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:50.811603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb3ee5-c274-4610-abcd-e3bf697075ad admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:50.822587 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:50.918064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1173/2340] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:47:50 2022] POST /identity/v3/projects => generated 377 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:47:50.922098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49c987-54b5-43c4-8d8e-3d82669c5e42 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:50.934527 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49c987-54b5-43c4-8d8e-3d82669c5e42 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:50.996972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56809fa-ee87-4f19-9bb5-fa64222a3842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:50.997650 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56809fa-ee87-4f19-9bb5-fa64222a3842 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:50.997935 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56809fa-ee87-4f19-9bb5-fa64222a3842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:50.998216 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56809fa-ee87-4f19-9bb5-fa64222a3842 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:50.999065 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56809fa-ee87-4f19-9bb5-fa64222a3842 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:51.001007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56809fa-ee87-4f19-9bb5-fa64222a3842 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:51.036175 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e4446-e6ce-44ef-8dfa-dd872c7f2bd9 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:51.036468 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e4446-e6ce-44ef-8dfa-dd872c7f2bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:51.036747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e4446-e6ce-44ef-8dfa-dd872c7f2bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:51.063641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-919e4446-e6ce-44ef-8dfa-dd872c7f2bd9 None None] MFA Rules not processed for user `f5f17909688f40f9b752e63b4074c2ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:51.065181 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-919e4446-e6ce-44ef-8dfa-dd872c7f2bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:51.114911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1168/2341] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:47:50 2022] POST /identity/v3/users => generated 295 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:47:51.119759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56809fa-ee87-4f19-9bb5-fa64222a3842 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:51.135974 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56809fa-ee87-4f19-9bb5-fa64222a3842 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:51.140982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024f1bf-30b4-4a3e-81a5-44d56c8dffe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:51.141579 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024f1bf-30b4-4a3e-81a5-44d56c8dffe0 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:51.141863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024f1bf-30b4-4a3e-81a5-44d56c8dffe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:51.142134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024f1bf-30b4-4a3e-81a5-44d56c8dffe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:51.142999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9024f1bf-30b4-4a3e-81a5-44d56c8dffe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:51.144619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9024f1bf-30b4-4a3e-81a5-44d56c8dffe0 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:51.151969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1174/2342] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:47:51.157864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1169/2343] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:47:51.162826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e4446-e6ce-44ef-8dfa-dd872c7f2bd9 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:51.184148 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e4446-e6ce-44ef-8dfa-dd872c7f2bd9 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:51.188620 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08792eb-c755-4ff6-8ae0-beb75808a72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:51.189522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08792eb-c755-4ff6-8ae0-beb75808a72e admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:51.189804 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08792eb-c755-4ff6-8ae0-beb75808a72e admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:51.190789 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08792eb-c755-4ff6-8ae0-beb75808a72e admin admin] PATH_INFO: `/v3/projects/a450aae6f84b437389d39783716ffb0d/users/c611129f5d1f4682840a158f127baf2a/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:51.213177 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08792eb-c755-4ff6-8ae0-beb75808a72e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a450aae6f84b437389d39783716ffb0d, user_id=c611129f5d1f4682840a158f127baf2a, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:51.217025 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08792eb-c755-4ff6-8ae0-beb75808a72e admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:51.302734 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1175/2344] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:47:51 2022] PUT /identity/v3/projects/a450aae6f84b437389d39783716ffb0d/users/c611129f5d1f4682840a158f127baf2a/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:47:51.307319 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024f1bf-30b4-4a3e-81a5-44d56c8dffe0 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:51.318287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024f1bf-30b4-4a3e-81a5-44d56c8dffe0 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:51.338801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:51.399174 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bf01f-09e0-4df4-a1ff-f16405c864b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:51.400058 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bf01f-09e0-4df4-a1ff-f16405c864b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:51.400436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bf01f-09e0-4df4-a1ff-f16405c864b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:51.400834 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bf01f-09e0-4df4-a1ff-f16405c864b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:51.401710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bf01f-09e0-4df4-a1ff-f16405c864b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:51.404146 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bf01f-09e0-4df4-a1ff-f16405c864b1 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:51.419521 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1170/2345] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:47:51.420710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08792eb-c755-4ff6-8ae0-beb75808a72e admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:51.434361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08792eb-c755-4ff6-8ae0-beb75808a72e admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:51.439188 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075ec0c-a2db-47d3-8f53-187d6ec73226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:51.440302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075ec0c-a2db-47d3-8f53-187d6ec73226 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:51.440427 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075ec0c-a2db-47d3-8f53-187d6ec73226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:51.440614 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075ec0c-a2db-47d3-8f53-187d6ec73226 admin admin] PATH_INFO: `/v3/projects/a450aae6f84b437389d39783716ffb0d/users/c611129f5d1f4682840a158f127baf2a/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:51.442767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b075ec0c-a2db-47d3-8f53-187d6ec73226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a450aae6f84b437389d39783716ffb0d, user_id=c611129f5d1f4682840a158f127baf2a, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:51.446051 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b075ec0c-a2db-47d3-8f53-187d6ec73226 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:51.456858 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1176/2346] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:47:51 2022] PUT /identity/v3/projects/a450aae6f84b437389d39783716ffb0d/users/c611129f5d1f4682840a158f127baf2a/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:47:51.468113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cb08b-5f2e-4182-9fcb-55da24fb5607 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:51.468542 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cb08b-5f2e-4182-9fcb-55da24fb5607 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:51.468996 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818cb08b-5f2e-4182-9fcb-55da24fb5607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:51.501345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-818cb08b-5f2e-4182-9fcb-55da24fb5607 None None] MFA Rules not processed for user `c611129f5d1f4682840a158f127baf2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:47:51.502720 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-818cb08b-5f2e-4182-9fcb-55da24fb5607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:51.595968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1171/2347] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:47:51 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:47:51.608915 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075ec0c-a2db-47d3-8f53-187d6ec73226 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:51.611068 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-b075ec0c-a2db-47d3-8f53-187d6ec73226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:47:51.677494 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075ec0c-a2db-47d3-8f53-187d6ec73226 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:51.682679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90746b-e878-4b91-bf50-83ad22e4bd68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:51.683430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90746b-e878-4b91-bf50-83ad22e4bd68 service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:51.683971 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90746b-e878-4b91-bf50-83ad22e4bd68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:51.684265 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90746b-e878-4b91-bf50-83ad22e4bd68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:51.752213 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce90746b-e878-4b91-bf50-83ad22e4bd68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:51.753679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce90746b-e878-4b91-bf50-83ad22e4bd68 service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:51.763839 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1177/2348] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:47:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:47:53.934779 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818cb08b-5f2e-4182-9fcb-55da24fb5607 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:53.946362 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818cb08b-5f2e-4182-9fcb-55da24fb5607 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:53.950196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404aa2ec-fcc1-42c7-98a6-c7d8dd8631a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:53.950784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404aa2ec-fcc1-42c7-98a6-c7d8dd8631a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:53.951050 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404aa2ec-fcc1-42c7-98a6-c7d8dd8631a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:53.951327 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404aa2ec-fcc1-42c7-98a6-c7d8dd8631a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:54.018240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404aa2ec-fcc1-42c7-98a6-c7d8dd8631a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:54.019667 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404aa2ec-fcc1-42c7-98a6-c7d8dd8631a2 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:54.029701 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1172/2349] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:47:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:47:55.491721 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90746b-e878-4b91-bf50-83ad22e4bd68 service cinder] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:47:55.503536 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90746b-e878-4b91-bf50-83ad22e4bd68 service cinder] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:47:55.514467 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ffe52-cf96-466d-9685-d4e242a828da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:47:55.515121 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ffe52-cf96-466d-9685-d4e242a828da service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:47:55.515399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ffe52-cf96-466d-9685-d4e242a828da service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:47:55.515673 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ffe52-cf96-466d-9685-d4e242a828da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:47:55.527063 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ffe52-cf96-466d-9685-d4e242a828da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:47:55.528556 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ffe52-cf96-466d-9685-d4e242a828da service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:47:55.533252 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:47:55.540995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1178/2350] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:47:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:48:00.152459 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404aa2ec-fcc1-42c7-98a6-c7d8dd8631a2 service cinder] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:00.197461 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:48:00.218974 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404aa2ec-fcc1-42c7-98a6-c7d8dd8631a2 service cinder] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:00.223585 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c829b0-244e-4aa9-b87e-512131f56384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:00.224677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c829b0-244e-4aa9-b87e-512131f56384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:00.224958 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c829b0-244e-4aa9-b87e-512131f56384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:00.225236 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c829b0-244e-4aa9-b87e-512131f56384 admin admin] PATH_INFO: `/v3/users/7dbe40e2b8d548d4a6226e171f016495` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:00.226627 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c829b0-244e-4aa9-b87e-512131f56384 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dbe40e2b8d548d4a6226e171f016495)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:00.228521 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c829b0-244e-4aa9-b87e-512131f56384 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:00.425223 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-92c829b0-244e-4aa9-b87e-512131f56384 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dbe40e2b8d548d4a6226e171f016495'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:00.433361 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-92c829b0-244e-4aa9-b87e-512131f56384 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dbe40e2b8d548d4a6226e171f016495'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:00.434106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-92c829b0-244e-4aa9-b87e-512131f56384 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dbe40e2b8d548d4a6226e171f016495'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:00.447300 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-92c829b0-244e-4aa9-b87e-512131f56384 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dbe40e2b8d548d4a6226e171f016495'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:00.513946 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-92c829b0-244e-4aa9-b87e-512131f56384 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dbe40e2b8d548d4a6226e171f016495'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:00.520389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1173/2351] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:48:00 2022] DELETE /identity/v3/users/7dbe40e2b8d548d4a6226e171f016495 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:48:00.525495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd117c34-2f60-42ae-947e-e139dbb2a085 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:00.525941 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd117c34-2f60-42ae-947e-e139dbb2a085 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:00.526140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd117c34-2f60-42ae-947e-e139dbb2a085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:00.544816 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-bd117c34-2f60-42ae-947e-e139dbb2a085 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:48:00.546059 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-bd117c34-2f60-42ae-947e-e139dbb2a085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:00.604714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1179/2352] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:48:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:48:00.614203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c829b0-244e-4aa9-b87e-512131f56384 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:00.615794 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-92c829b0-244e-4aa9-b87e-512131f56384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:00.673362 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c829b0-244e-4aa9-b87e-512131f56384 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:00.677031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c5600-78ef-47f3-870e-201a2397f197 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:00.677659 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c5600-78ef-47f3-870e-201a2397f197 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:00.677939 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c5600-78ef-47f3-870e-201a2397f197 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:00.678214 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c5600-78ef-47f3-870e-201a2397f197 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:00.687540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c5600-78ef-47f3-870e-201a2397f197 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:00.688958 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c5600-78ef-47f3-870e-201a2397f197 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:00.699478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1174/2353] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:48:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:48:01.190540 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd117c34-2f60-42ae-947e-e139dbb2a085 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:01.198847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-bd117c34-2f60-42ae-947e-e139dbb2a085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:01.211687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd117c34-2f60-42ae-947e-e139dbb2a085 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:01.216622 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbcdc5-f910-4234-aa5f-54838ae60ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:01.217479 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbcdc5-f910-4234-aa5f-54838ae60ff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:01.217764 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbcdc5-f910-4234-aa5f-54838ae60ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:01.218042 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acbcdc5-f910-4234-aa5f-54838ae60ff4 admin admin] PATH_INFO: `/v3/projects/d7ecbf4573cc4b57b7564fa10664d7be` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:01.219273 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbcdc5-f910-4234-aa5f-54838ae60ff4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7ecbf4573cc4b57b7564fa10664d7be)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:01.221193 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acbcdc5-f910-4234-aa5f-54838ae60ff4 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:01.353501 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-8acbcdc5-f910-4234-aa5f-54838ae60ff4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7ecbf4573cc4b57b7564fa10664d7be', 'request_id': 'req-8acbcdc5-f910-4234-aa5f-54838ae60ff4'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:01.457081 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-8acbcdc5-f910-4234-aa5f-54838ae60ff4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7ecbf4573cc4b57b7564fa10664d7be', 'request_id': 'req-8acbcdc5-f910-4234-aa5f-54838ae60ff4'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:01.464484 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-8acbcdc5-f910-4234-aa5f-54838ae60ff4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7ecbf4573cc4b57b7564fa10664d7be', 'request_id': 'req-8acbcdc5-f910-4234-aa5f-54838ae60ff4'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:01.475041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-8acbcdc5-f910-4234-aa5f-54838ae60ff4 admin admin] The token cache is being invalidate because project d7ecbf4573cc4b57b7564fa10664d7be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:48:01.475333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-8acbcdc5-f910-4234-aa5f-54838ae60ff4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:01.475985 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-8acbcdc5-f910-4234-aa5f-54838ae60ff4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:01.477803 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1180/2354] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:48:01 2022] DELETE /identity/v3/projects/d7ecbf4573cc4b57b7564fa10664d7be => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:48:01.482139 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c5600-78ef-47f3-870e-201a2397f197 service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:01.492096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c5600-78ef-47f3-870e-201a2397f197 service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:01.549230 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110cee6b-38bb-40bb-a585-28c7b06a8df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:01.550117 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cee6b-38bb-40bb-a585-28c7b06a8df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:01.550546 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cee6b-38bb-40bb-a585-28c7b06a8df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:01.550758 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cee6b-38bb-40bb-a585-28c7b06a8df5 admin admin] PATH_INFO: `/v3/users/d3e12bc3c2c4478c89b17bd249befd0d` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:01.551967 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110cee6b-38bb-40bb-a585-28c7b06a8df5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3e12bc3c2c4478c89b17bd249befd0d)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:01.554267 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110cee6b-38bb-40bb-a585-28c7b06a8df5 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:01.556512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:48:01.732570 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-110cee6b-38bb-40bb-a585-28c7b06a8df5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3e12bc3c2c4478c89b17bd249befd0d'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:01.740130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-110cee6b-38bb-40bb-a585-28c7b06a8df5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3e12bc3c2c4478c89b17bd249befd0d'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:01.740849 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-110cee6b-38bb-40bb-a585-28c7b06a8df5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3e12bc3c2c4478c89b17bd249befd0d'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:01.761173 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-110cee6b-38bb-40bb-a585-28c7b06a8df5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3e12bc3c2c4478c89b17bd249befd0d'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:01.857001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-110cee6b-38bb-40bb-a585-28c7b06a8df5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3e12bc3c2c4478c89b17bd249befd0d'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:01.864597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1175/2355] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:48:01 2022] DELETE /identity/v3/users/d3e12bc3c2c4478c89b17bd249befd0d => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:48:02.398262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbcdc5-f910-4234-aa5f-54838ae60ff4 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:02.400440 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-8acbcdc5-f910-4234-aa5f-54838ae60ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:02.421547 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acbcdc5-f910-4234-aa5f-54838ae60ff4 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:02.483054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad884ad-7b9a-4a82-9d15-5dbf8aab0b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:02.483907 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad884ad-7b9a-4a82-9d15-5dbf8aab0b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:02.484185 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad884ad-7b9a-4a82-9d15-5dbf8aab0b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:02.484455 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad884ad-7b9a-4a82-9d15-5dbf8aab0b02 admin admin] PATH_INFO: `/v3/projects/d5e39794c15d4c5ea19569dc4d6de4c9` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:02.485630 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad884ad-7b9a-4a82-9d15-5dbf8aab0b02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5e39794c15d4c5ea19569dc4d6de4c9)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:02.487559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad884ad-7b9a-4a82-9d15-5dbf8aab0b02 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:02.512646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:48:02.599802 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-6ad884ad-7b9a-4a82-9d15-5dbf8aab0b02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5e39794c15d4c5ea19569dc4d6de4c9', 'request_id': 'req-6ad884ad-7b9a-4a82-9d15-5dbf8aab0b02'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:02.712218 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-6ad884ad-7b9a-4a82-9d15-5dbf8aab0b02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5e39794c15d4c5ea19569dc4d6de4c9', 'request_id': 'req-6ad884ad-7b9a-4a82-9d15-5dbf8aab0b02'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:02.718983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-6ad884ad-7b9a-4a82-9d15-5dbf8aab0b02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5e39794c15d4c5ea19569dc4d6de4c9', 'request_id': 'req-6ad884ad-7b9a-4a82-9d15-5dbf8aab0b02'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:02.730565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-6ad884ad-7b9a-4a82-9d15-5dbf8aab0b02 admin admin] The token cache is being invalidate because project d5e39794c15d4c5ea19569dc4d6de4c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:48:02.730855 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-6ad884ad-7b9a-4a82-9d15-5dbf8aab0b02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:02.731502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-6ad884ad-7b9a-4a82-9d15-5dbf8aab0b02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:02.733045 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1181/2356] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:48:02 2022] DELETE /identity/v3/projects/d5e39794c15d4c5ea19569dc4d6de4c9 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:48:02.740253 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110cee6b-38bb-40bb-a585-28c7b06a8df5 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:02.740253 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-110cee6b-38bb-40bb-a585-28c7b06a8df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:02.751963 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110cee6b-38bb-40bb-a585-28c7b06a8df5 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:02.811356 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be0850-f08d-4d9b-9b28-9ab76980f3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:02.812231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be0850-f08d-4d9b-9b28-9ab76980f3fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:02.812514 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be0850-f08d-4d9b-9b28-9ab76980f3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:02.812800 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be0850-f08d-4d9b-9b28-9ab76980f3fa admin admin] PATH_INFO: `/v3/users/c611129f5d1f4682840a158f127baf2a` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:02.814006 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be0850-f08d-4d9b-9b28-9ab76980f3fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c611129f5d1f4682840a158f127baf2a)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:02.815993 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be0850-f08d-4d9b-9b28-9ab76980f3fa admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:03.013174 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-16be0850-f08d-4d9b-9b28-9ab76980f3fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c611129f5d1f4682840a158f127baf2a'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:03.019930 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-16be0850-f08d-4d9b-9b28-9ab76980f3fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c611129f5d1f4682840a158f127baf2a'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:03.020623 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-16be0850-f08d-4d9b-9b28-9ab76980f3fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c611129f5d1f4682840a158f127baf2a'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:03.033559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-16be0850-f08d-4d9b-9b28-9ab76980f3fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c611129f5d1f4682840a158f127baf2a'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:03.108015 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-16be0850-f08d-4d9b-9b28-9ab76980f3fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c611129f5d1f4682840a158f127baf2a'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:03.109542 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:48:03.114899 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1176/2357] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:48:02 2022] DELETE /identity/v3/users/c611129f5d1f4682840a158f127baf2a => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:48:03.568287 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad884ad-7b9a-4a82-9d15-5dbf8aab0b02 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:03.569867 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-6ad884ad-7b9a-4a82-9d15-5dbf8aab0b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:03.585318 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad884ad-7b9a-4a82-9d15-5dbf8aab0b02 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:03.649533 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a6398-9f33-4633-800e-f12accce67e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:03.650435 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a6398-9f33-4633-800e-f12accce67e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:03.650726 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a6398-9f33-4633-800e-f12accce67e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:03.651004 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a6398-9f33-4633-800e-f12accce67e7 admin admin] PATH_INFO: `/v3/projects/a450aae6f84b437389d39783716ffb0d` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:03.652220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076a6398-9f33-4633-800e-f12accce67e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a450aae6f84b437389d39783716ffb0d)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:03.654423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076a6398-9f33-4633-800e-f12accce67e7 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:03.787165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-076a6398-9f33-4633-800e-f12accce67e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a450aae6f84b437389d39783716ffb0d', 'request_id': 'req-076a6398-9f33-4633-800e-f12accce67e7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:03.854902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-076a6398-9f33-4633-800e-f12accce67e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a450aae6f84b437389d39783716ffb0d', 'request_id': 'req-076a6398-9f33-4633-800e-f12accce67e7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:03.861678 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-076a6398-9f33-4633-800e-f12accce67e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a450aae6f84b437389d39783716ffb0d', 'request_id': 'req-076a6398-9f33-4633-800e-f12accce67e7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:03.872553 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-076a6398-9f33-4633-800e-f12accce67e7 admin admin] The token cache is being invalidate because project a450aae6f84b437389d39783716ffb0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:48:03.872840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-076a6398-9f33-4633-800e-f12accce67e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:03.876120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-076a6398-9f33-4633-800e-f12accce67e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:03.876120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1182/2358] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:48:03 2022] DELETE /identity/v3/projects/a450aae6f84b437389d39783716ffb0d => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:48:03.880124 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bb9b3-e657-4e9e-909a-9202b70d0ad3 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:03.880580 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bb9b3-e657-4e9e-909a-9202b70d0ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:03.880689 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bb9b3-e657-4e9e-909a-9202b70d0ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:03.900000 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-0b2bb9b3-e657-4e9e-909a-9202b70d0ad3 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:48:03.901467 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-0b2bb9b3-e657-4e9e-909a-9202b70d0ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:03.971904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1177/2359] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:48:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:48:03.981350 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ad815-64fd-4c0d-b7c6-ab8533319470 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:03.981642 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ad815-64fd-4c0d-b7c6-ab8533319470 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:03.981924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ad815-64fd-4c0d-b7c6-ab8533319470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:04.004244 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-410ad815-64fd-4c0d-b7c6-ab8533319470 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:48:04.005664 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-410ad815-64fd-4c0d-b7c6-ab8533319470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:04.015892 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1183/2360] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:48:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:48:04.019766 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bb9b3-e657-4e9e-909a-9202b70d0ad3 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:04.030113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2bb9b3-e657-4e9e-909a-9202b70d0ad3 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:04.034959 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd005c7-6881-4246-abb8-49d9d579db0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:04.035696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd005c7-6881-4246-abb8-49d9d579db0c admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:04.036049 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd005c7-6881-4246-abb8-49d9d579db0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:04.036399 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd005c7-6881-4246-abb8-49d9d579db0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:04.037212 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd005c7-6881-4246-abb8-49d9d579db0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:48:04.037585 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd005c7-6881-4246-abb8-49d9d579db0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:04.039427 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd005c7-6881-4246-abb8-49d9d579db0c admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:04.054757 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1178/2361] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:48:04.057051 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ad815-64fd-4c0d-b7c6-ab8533319470 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:04.066481 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ad815-64fd-4c0d-b7c6-ab8533319470 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:04.071178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b5b731-ce64-4b96-8957-74969f05a3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:04.071824 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b5b731-ce64-4b96-8957-74969f05a3a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:04.072104 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b5b731-ce64-4b96-8957-74969f05a3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:04.072378 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b5b731-ce64-4b96-8957-74969f05a3a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:04.073095 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b5b731-ce64-4b96-8957-74969f05a3a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:04.075035 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b5b731-ce64-4b96-8957-74969f05a3a8 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:04.170714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1184/2362] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:48:04 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:48:04.172616 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd005c7-6881-4246-abb8-49d9d579db0c admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:04.183706 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd005c7-6881-4246-abb8-49d9d579db0c admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:04.240080 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04242de9-74a4-43df-971c-9bf04542d203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:04.240729 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04242de9-74a4-43df-971c-9bf04542d203 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:04.241010 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04242de9-74a4-43df-971c-9bf04542d203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:04.241283 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04242de9-74a4-43df-971c-9bf04542d203 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:04.241957 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04242de9-74a4-43df-971c-9bf04542d203 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:04.244325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04242de9-74a4-43df-971c-9bf04542d203 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:04.280578 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd8347-db34-45c5-a911-d2964743eeda None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:04.280873 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd8347-db34-45c5-a911-d2964743eeda None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:04.281153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd8347-db34-45c5-a911-d2964743eeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:04.306165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:48:04.332753 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-c0dd8347-db34-45c5-a911-d2964743eeda None None] MFA Rules not processed for user `58e1f1ac6ea14da0be6e9a24d007847a`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:48:04.333968 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c0dd8347-db34-45c5-a911-d2964743eeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:04.397360 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1185/2363] 10.222.0.15 () {58 vars in 967 bytes} [Mon Jun 27 18:48:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:48:04.410190 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd8347-db34-45c5-a911-d2964743eeda None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:04.411670 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c0dd8347-db34-45c5-a911-d2964743eeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:04.414910 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1179/2364] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:48:04 2022] POST /identity/v3/users => generated 287 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:48:04.416883 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04242de9-74a4-43df-971c-9bf04542d203 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:04.427305 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04242de9-74a4-43df-971c-9bf04542d203 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:04.432728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858180ab-7d53-4769-982b-449c25184eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:04.433433 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858180ab-7d53-4769-982b-449c25184eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:04.433716 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858180ab-7d53-4769-982b-449c25184eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:04.433993 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858180ab-7d53-4769-982b-449c25184eb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:04.434781 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858180ab-7d53-4769-982b-449c25184eb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:04.436466 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858180ab-7d53-4769-982b-449c25184eb5 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:04.447221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1180/2365] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:48:04.451104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858180ab-7d53-4769-982b-449c25184eb5 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:04.463488 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858180ab-7d53-4769-982b-449c25184eb5 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:04.468486 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03030690-6d62-439a-95dc-cfe6fe9a5bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:04.469156 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03030690-6d62-439a-95dc-cfe6fe9a5bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:04.469441 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03030690-6d62-439a-95dc-cfe6fe9a5bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:04.469717 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03030690-6d62-439a-95dc-cfe6fe9a5bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:04.470492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03030690-6d62-439a-95dc-cfe6fe9a5bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:04.472126 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03030690-6d62-439a-95dc-cfe6fe9a5bc2 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:04.484824 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1181/2366] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:48:04.488789 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03030690-6d62-439a-95dc-cfe6fe9a5bc2 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:04.499483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03030690-6d62-439a-95dc-cfe6fe9a5bc2 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:04.504432 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdf24a-2822-4ce4-8ecf-18f506a566d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:04.505302 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdf24a-2822-4ce4-8ecf-18f506a566d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:04.505586 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdf24a-2822-4ce4-8ecf-18f506a566d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:04.505869 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdf24a-2822-4ce4-8ecf-18f506a566d9 admin admin] PATH_INFO: `/v3/projects/bc7a3d93a3a84224b7b23d93924f276f/users/d3c1ad9da81f48c390ad82e10e44712d/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:04.510402 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd8347-db34-45c5-a911-d2964743eeda None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:04.515715 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ac9f3-43c2-4f52-b4ed-d67e96e6f509 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1529c9a45434a19968e924797688f06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:04.516380 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ac9f3-43c2-4f52-b4ed-d67e96e6f509 service swift] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:04.516665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ac9f3-43c2-4f52-b4ed-d67e96e6f509 service swift] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:04.516942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ac9f3-43c2-4f52-b4ed-d67e96e6f509 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:04.527662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdf24a-2822-4ce4-8ecf-18f506a566d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc7a3d93a3a84224b7b23d93924f276f, user_id=d3c1ad9da81f48c390ad82e10e44712d, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:04.527786 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ac9f3-43c2-4f52-b4ed-d67e96e6f509 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:04.527884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdf24a-2822-4ce4-8ecf-18f506a566d9 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:04.527965 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ac9f3-43c2-4f52-b4ed-d67e96e6f509 service swift] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:04.535905 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1186/2367] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Jun 27 18:48:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 18:48:04.595663 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1182/2368] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:48:04 2022] PUT /identity/v3/projects/bc7a3d93a3a84224b7b23d93924f276f/users/d3c1ad9da81f48c390ad82e10e44712d/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:48:04.600703 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523c92e-1700-4e71-abd9-8df1c44f29cb None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:04.601178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523c92e-1700-4e71-abd9-8df1c44f29cb None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:04.601297 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523c92e-1700-4e71-abd9-8df1c44f29cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:04.621313 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-c523c92e-1700-4e71-abd9-8df1c44f29cb None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:48:04.622528 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c523c92e-1700-4e71-abd9-8df1c44f29cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:04.680983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1187/2369] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:48:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:48:04.690495 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdf24a-2822-4ce4-8ecf-18f506a566d9 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:04.692146 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-b6bdf24a-2822-4ce4-8ecf-18f506a566d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:04.731953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:48:04.767776 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdf24a-2822-4ce4-8ecf-18f506a566d9 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:04.775400 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c425b-322e-4eb7-837b-78aef34886fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:04.776191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c425b-322e-4eb7-837b-78aef34886fb service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:04.776591 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c425b-322e-4eb7-837b-78aef34886fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:04.776973 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c425b-322e-4eb7-837b-78aef34886fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:04.787535 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581c425b-322e-4eb7-837b-78aef34886fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:04.789249 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581c425b-322e-4eb7-837b-78aef34886fb service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:04.802209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1183/2370] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:48:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:48:04.844061 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:48:06.450824 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef895d0-c7a7-41d8-85f1-3595daa01c6b None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:06.451780 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef895d0-c7a7-41d8-85f1-3595daa01c6b None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:06.452244 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef895d0-c7a7-41d8-85f1-3595daa01c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:06.487048 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-3ef895d0-c7a7-41d8-85f1-3595daa01c6b None None] MFA Rules not processed for user `d3c1ad9da81f48c390ad82e10e44712d`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:48:06.488329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-3ef895d0-c7a7-41d8-85f1-3595daa01c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:06.574281 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1188/2371] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:48:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:48:06.582716 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c425b-322e-4eb7-837b-78aef34886fb service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:06.584165 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-581c425b-322e-4eb7-837b-78aef34886fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:06.642638 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c425b-322e-4eb7-837b-78aef34886fb service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:06.646212 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f07769-e581-4109-a520-e2d6562116df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:06.646952 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f07769-e581-4109-a520-e2d6562116df service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:06.647238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f07769-e581-4109-a520-e2d6562116df service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:06.647571 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f07769-e581-4109-a520-e2d6562116df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:06.656153 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f07769-e581-4109-a520-e2d6562116df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:06.657582 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f07769-e581-4109-a520-e2d6562116df service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:06.667381 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1184/2372] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:48:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:48:08.436233 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea33d2-bec9-46f2-80c9-eceb94a6fc6a None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:08.436233 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea33d2-bec9-46f2-80c9-eceb94a6fc6a None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:08.436233 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea33d2-bec9-46f2-80c9-eceb94a6fc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:08.460172 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-6eea33d2-bec9-46f2-80c9-eceb94a6fc6a None None] MFA Rules not processed for user `d3c1ad9da81f48c390ad82e10e44712d`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:48:08.461718 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-6eea33d2-bec9-46f2-80c9-eceb94a6fc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:08.469942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1189/2373] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:48:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:48:14.323556 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f07769-e581-4109-a520-e2d6562116df service cinder] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:14.325049 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-53f07769-e581-4109-a520-e2d6562116df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:14.424141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f07769-e581-4109-a520-e2d6562116df service cinder] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:14.429098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec3514-df62-45e2-98ef-493c8d1716e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cd6f0a1286243e4add692db01028205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:14.429740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec3514-df62-45e2-98ef-493c8d1716e9 service placement] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:14.430019 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec3514-df62-45e2-98ef-493c8d1716e9 service placement] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:14.430291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec3514-df62-45e2-98ef-493c8d1716e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:14.433098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-02ec3514-df62-45e2-98ef-493c8d1716e9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:14.445169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec3514-df62-45e2-98ef-493c8d1716e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:14.447407 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec3514-df62-45e2-98ef-493c8d1716e9 service placement] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:14.458356 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1185/2374] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:48:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:48:18.343629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea33d2-bec9-46f2-80c9-eceb94a6fc6a None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:18.345149 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-6eea33d2-bec9-46f2-80c9-eceb94a6fc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:18.358820 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea33d2-bec9-46f2-80c9-eceb94a6fc6a None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:18.363319 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e360140-196d-481e-8db1-cdde4a1f2557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:18.364263 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e360140-196d-481e-8db1-cdde4a1f2557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:18.364545 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e360140-196d-481e-8db1-cdde4a1f2557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:18.364817 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e360140-196d-481e-8db1-cdde4a1f2557 admin admin] PATH_INFO: `/v3/users/63441402d47444d98d9ac91488a08aa1` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:18.365994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e360140-196d-481e-8db1-cdde4a1f2557 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63441402d47444d98d9ac91488a08aa1)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:18.368588 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e360140-196d-481e-8db1-cdde4a1f2557 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:18.538821 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-4e360140-196d-481e-8db1-cdde4a1f2557 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63441402d47444d98d9ac91488a08aa1'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:18.546437 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-4e360140-196d-481e-8db1-cdde4a1f2557 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63441402d47444d98d9ac91488a08aa1'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:18.547096 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-4e360140-196d-481e-8db1-cdde4a1f2557 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63441402d47444d98d9ac91488a08aa1'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:18.547708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-4e360140-196d-481e-8db1-cdde4a1f2557 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63441402d47444d98d9ac91488a08aa1'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:18.560189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-4e360140-196d-481e-8db1-cdde4a1f2557 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63441402d47444d98d9ac91488a08aa1'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:18.619999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1190/2375] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:48:18 2022] DELETE /identity/v3/users/63441402d47444d98d9ac91488a08aa1 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:48:18.624474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bb185-249d-470c-bf1b-3c1d8cc94c5b None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:18.628842 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bb185-249d-470c-bf1b-3c1d8cc94c5b None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:18.628842 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bb185-249d-470c-bf1b-3c1d8cc94c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:18.652286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-439bb185-249d-470c-bf1b-3c1d8cc94c5b None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:48:18.656117 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-439bb185-249d-470c-bf1b-3c1d8cc94c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:18.701390 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:48:18.721098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1186/2376] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:48:18.731320 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e360140-196d-481e-8db1-cdde4a1f2557 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:18.732759 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-4e360140-196d-481e-8db1-cdde4a1f2557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:18.809170 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e360140-196d-481e-8db1-cdde4a1f2557 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:18.813957 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8786e14-ff99-4b1b-a6f8-ae9e1f3a4a38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:18.819038 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8786e14-ff99-4b1b-a6f8-ae9e1f3a4a38 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:18.819359 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8786e14-ff99-4b1b-a6f8-ae9e1f3a4a38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:18.819618 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8786e14-ff99-4b1b-a6f8-ae9e1f3a4a38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:18.827701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:48:18.831783 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8786e14-ff99-4b1b-a6f8-ae9e1f3a4a38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:18.833257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8786e14-ff99-4b1b-a6f8-ae9e1f3a4a38 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:18.844263 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1191/2377] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:48:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:48:19.275417 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bb185-249d-470c-bf1b-3c1d8cc94c5b None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:19.277031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-439bb185-249d-470c-bf1b-3c1d8cc94c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:19.288701 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bb185-249d-470c-bf1b-3c1d8cc94c5b None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:19.295007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147616c-e3d8-4eff-8606-c3416f226b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:19.295868 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147616c-e3d8-4eff-8606-c3416f226b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:19.296149 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147616c-e3d8-4eff-8606-c3416f226b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:19.296418 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147616c-e3d8-4eff-8606-c3416f226b3c admin admin] PATH_INFO: `/v3/projects/2e8fc4b97c74407bbf235ef08f8797d2` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:19.297589 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8147616c-e3d8-4eff-8606-c3416f226b3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e8fc4b97c74407bbf235ef08f8797d2)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:19.299525 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8147616c-e3d8-4eff-8606-c3416f226b3c admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:19.435673 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-8147616c-e3d8-4eff-8606-c3416f226b3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e8fc4b97c74407bbf235ef08f8797d2', 'request_id': 'req-8147616c-e3d8-4eff-8606-c3416f226b3c'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:19.517049 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-8147616c-e3d8-4eff-8606-c3416f226b3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e8fc4b97c74407bbf235ef08f8797d2', 'request_id': 'req-8147616c-e3d8-4eff-8606-c3416f226b3c'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:19.523755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-8147616c-e3d8-4eff-8606-c3416f226b3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e8fc4b97c74407bbf235ef08f8797d2', 'request_id': 'req-8147616c-e3d8-4eff-8606-c3416f226b3c'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:19.534230 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-8147616c-e3d8-4eff-8606-c3416f226b3c admin admin] The token cache is being invalidate because project 2e8fc4b97c74407bbf235ef08f8797d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:48:19.534543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-8147616c-e3d8-4eff-8606-c3416f226b3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:19.535171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-8147616c-e3d8-4eff-8606-c3416f226b3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:48:19.536719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1187/2378] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:48:19 2022] DELETE /identity/v3/projects/2e8fc4b97c74407bbf235ef08f8797d2 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:48:19.542183 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26affe-8ca6-41da-aa2c-930485049504 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:19.542711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26affe-8ca6-41da-aa2c-930485049504 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:19.542995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26affe-8ca6-41da-aa2c-930485049504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:19.561411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-9b26affe-8ca6-41da-aa2c-930485049504 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:48:19.562794 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-9b26affe-8ca6-41da-aa2c-930485049504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:19.629973 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1192/2379] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:48:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:48:19.638496 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a060ae2-006d-4b63-90dd-4b343e0a659f None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:19.638974 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a060ae2-006d-4b63-90dd-4b343e0a659f None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:19.639079 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a060ae2-006d-4b63-90dd-4b343e0a659f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:19.657178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-6a060ae2-006d-4b63-90dd-4b343e0a659f None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:48:19.658402 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-6a060ae2-006d-4b63-90dd-4b343e0a659f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:19.666434 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1188/2380] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:48:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:48:19.670372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26affe-8ca6-41da-aa2c-930485049504 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:19.680077 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26affe-8ca6-41da-aa2c-930485049504 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:19.685125 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e59e6-5192-4154-9af1-ce3caab481ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:19.685771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e59e6-5192-4154-9af1-ce3caab481ea admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:19.686053 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e59e6-5192-4154-9af1-ce3caab481ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:19.686328 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e59e6-5192-4154-9af1-ce3caab481ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:19.687114 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e59e6-5192-4154-9af1-ce3caab481ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:48:19.687419 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e59e6-5192-4154-9af1-ce3caab481ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:19.689015 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e59e6-5192-4154-9af1-ce3caab481ea admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:19.700909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1193/2381] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:48:19.704923 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a060ae2-006d-4b63-90dd-4b343e0a659f None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:19.713980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a060ae2-006d-4b63-90dd-4b343e0a659f None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:19.718661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8a445-64c3-4ece-8f28-14357df31848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:19.719294 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8a445-64c3-4ece-8f28-14357df31848 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:19.719577 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8a445-64c3-4ece-8f28-14357df31848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:19.719855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8a445-64c3-4ece-8f28-14357df31848 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:19.720537 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f8a445-64c3-4ece-8f28-14357df31848 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:19.722448 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f8a445-64c3-4ece-8f28-14357df31848 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:19.837921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1189/2382] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:48:19 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:48:19.839116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e59e6-5192-4154-9af1-ce3caab481ea admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:19.849530 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e59e6-5192-4154-9af1-ce3caab481ea admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:19.918375 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1cd77-a4f8-40cd-81b2-4a1a2a013f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:19.919076 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1cd77-a4f8-40cd-81b2-4a1a2a013f05 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:19.919356 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1cd77-a4f8-40cd-81b2-4a1a2a013f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:19.919631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1cd77-a4f8-40cd-81b2-4a1a2a013f05 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:19.920336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e1cd77-a4f8-40cd-81b2-4a1a2a013f05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:19.923193 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e1cd77-a4f8-40cd-81b2-4a1a2a013f05 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:20.013223 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1194/2383] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:48:19 2022] POST /identity/v3/users => generated 287 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:48:20.024299 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8a445-64c3-4ece-8f28-14357df31848 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:20.044699 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8a445-64c3-4ece-8f28-14357df31848 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:20.056218 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d34dd5-35e2-4656-8836-cf9691b3c650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:20.056218 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d34dd5-35e2-4656-8836-cf9691b3c650 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:20.056492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d34dd5-35e2-4656-8836-cf9691b3c650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:20.056857 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d34dd5-35e2-4656-8836-cf9691b3c650 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:20.057684 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d34dd5-35e2-4656-8836-cf9691b3c650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:20.059535 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d34dd5-35e2-4656-8836-cf9691b3c650 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:20.073430 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1190/2384] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:48:20.077403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1cd77-a4f8-40cd-81b2-4a1a2a013f05 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:20.088092 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1cd77-a4f8-40cd-81b2-4a1a2a013f05 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:20.092793 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9906a2-1fcd-4293-a130-e780ea8959ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:20.093550 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9906a2-1fcd-4293-a130-e780ea8959ed admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:20.093839 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9906a2-1fcd-4293-a130-e780ea8959ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:20.094123 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9906a2-1fcd-4293-a130-e780ea8959ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:20.094970 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9906a2-1fcd-4293-a130-e780ea8959ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:20.097078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9906a2-1fcd-4293-a130-e780ea8959ed admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:20.115735 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1195/2385] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:48:20.120230 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d34dd5-35e2-4656-8836-cf9691b3c650 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:20.130999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d34dd5-35e2-4656-8836-cf9691b3c650 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:20.137494 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba057879-efa8-4a74-8a36-26b75a1884b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:20.140975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba057879-efa8-4a74-8a36-26b75a1884b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:20.140975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba057879-efa8-4a74-8a36-26b75a1884b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:20.140975 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba057879-efa8-4a74-8a36-26b75a1884b4 admin admin] PATH_INFO: `/v3/projects/61866ab089f04e9abfab3f6ce58cf6b8/users/49ccbcfbd48642aabd74d789b1875dd3/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:20.154287 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba057879-efa8-4a74-8a36-26b75a1884b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61866ab089f04e9abfab3f6ce58cf6b8, user_id=49ccbcfbd48642aabd74d789b1875dd3, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:20.156973 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba057879-efa8-4a74-8a36-26b75a1884b4 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:20.240293 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1191/2386] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:48:20 2022] PUT /identity/v3/projects/61866ab089f04e9abfab3f6ce58cf6b8/users/49ccbcfbd48642aabd74d789b1875dd3/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:48:20.246099 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1710ebe-366f-4c80-ac31-91d4f4eec919 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:20.246420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1710ebe-366f-4c80-ac31-91d4f4eec919 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:20.246705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1710ebe-366f-4c80-ac31-91d4f4eec919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:20.264391 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-e1710ebe-366f-4c80-ac31-91d4f4eec919 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:48:20.265675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-e1710ebe-366f-4c80-ac31-91d4f4eec919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:20.318718 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:48:20.327840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1196/2387] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:48:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:48:20.338499 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba057879-efa8-4a74-8a36-26b75a1884b4 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:20.340288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-ba057879-efa8-4a74-8a36-26b75a1884b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:20.397256 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba057879-efa8-4a74-8a36-26b75a1884b4 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:20.401816 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716e112-f1e5-4696-9c39-6c5c021e2952 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:20.402485 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716e112-f1e5-4696-9c39-6c5c021e2952 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:20.402769 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716e112-f1e5-4696-9c39-6c5c021e2952 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:20.403047 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716e112-f1e5-4696-9c39-6c5c021e2952 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:20.412891 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716e112-f1e5-4696-9c39-6c5c021e2952 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:20.414412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716e112-f1e5-4696-9c39-6c5c021e2952 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:20.425714 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1192/2388] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:48:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:48:25.942644 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec6bd1-2f69-44e9-bef5-32b142a10edf None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:25.943193 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec6bd1-2f69-44e9-bef5-32b142a10edf None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:25.943193 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec6bd1-2f69-44e9-bef5-32b142a10edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:25.982031 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-9aec6bd1-2f69-44e9-bef5-32b142a10edf None None] MFA Rules not processed for user `49ccbcfbd48642aabd74d789b1875dd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:48:25.983682 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-9aec6bd1-2f69-44e9-bef5-32b142a10edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:26.070809 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1197/2389] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:48:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:48:26.075676 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716e112-f1e5-4696-9c39-6c5c021e2952 service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:26.095656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716e112-f1e5-4696-9c39-6c5c021e2952 service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:26.100651 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6663b-7464-4f9c-9fc0-ae7e72a46c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:26.101424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6663b-7464-4f9c-9fc0-ae7e72a46c9f admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:26.102407 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6663b-7464-4f9c-9fc0-ae7e72a46c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:26.102941 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6663b-7464-4f9c-9fc0-ae7e72a46c9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:26.104122 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6663b-7464-4f9c-9fc0-ae7e72a46c9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:26.106083 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6663b-7464-4f9c-9fc0-ae7e72a46c9f admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:26.109673 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:48:26.194289 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1193/2390] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:48:26 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:48:26.198761 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec6bd1-2f69-44e9-bef5-32b142a10edf None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:26.210719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec6bd1-2f69-44e9-bef5-32b142a10edf None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:26.281944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0becdbdb-da67-444c-b8d7-0140a5de0b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:26.282686 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becdbdb-da67-444c-b8d7-0140a5de0b6e admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:26.282970 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becdbdb-da67-444c-b8d7-0140a5de0b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:26.283244 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0becdbdb-da67-444c-b8d7-0140a5de0b6e admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:26.283949 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0becdbdb-da67-444c-b8d7-0140a5de0b6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:26.285928 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0becdbdb-da67-444c-b8d7-0140a5de0b6e admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:26.375338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1198/2391] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:48:26 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:48:26.379223 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6663b-7464-4f9c-9fc0-ae7e72a46c9f admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:26.392030 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6663b-7464-4f9c-9fc0-ae7e72a46c9f admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:26.401853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01613315-6fe5-487f-8a0a-d0c3796d8de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:26.401853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01613315-6fe5-487f-8a0a-d0c3796d8de2 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:26.401853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01613315-6fe5-487f-8a0a-d0c3796d8de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:26.401853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01613315-6fe5-487f-8a0a-d0c3796d8de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:26.402618 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01613315-6fe5-487f-8a0a-d0c3796d8de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:26.404250 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01613315-6fe5-487f-8a0a-d0c3796d8de2 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:26.426838 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1194/2392] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:48:26.428341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0becdbdb-da67-444c-b8d7-0140a5de0b6e admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:26.440065 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0becdbdb-da67-444c-b8d7-0140a5de0b6e admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:26.446031 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d52d36-4a19-43ae-a3f6-4bb064e6b7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:26.447117 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d52d36-4a19-43ae-a3f6-4bb064e6b7b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:26.447479 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d52d36-4a19-43ae-a3f6-4bb064e6b7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:26.447647 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d52d36-4a19-43ae-a3f6-4bb064e6b7b1 admin admin] PATH_INFO: `/v3/projects/48eeb3fe7e194bf9aea90d74a07ec266/users/b4a8f980f48a4de2a3db6cfbf718beab/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:26.475861 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d52d36-4a19-43ae-a3f6-4bb064e6b7b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48eeb3fe7e194bf9aea90d74a07ec266, user_id=b4a8f980f48a4de2a3db6cfbf718beab, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:26.479730 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d52d36-4a19-43ae-a3f6-4bb064e6b7b1 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:26.547774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1199/2393] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:48:26 2022] PUT /identity/v3/projects/48eeb3fe7e194bf9aea90d74a07ec266/users/b4a8f980f48a4de2a3db6cfbf718beab/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:48:26.551689 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01613315-6fe5-487f-8a0a-d0c3796d8de2 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:26.562304 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01613315-6fe5-487f-8a0a-d0c3796d8de2 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:26.621260 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99f3da-3e06-4502-99ab-6be67c4982a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:26.621920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99f3da-3e06-4502-99ab-6be67c4982a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:26.622197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99f3da-3e06-4502-99ab-6be67c4982a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:26.622501 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99f3da-3e06-4502-99ab-6be67c4982a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:26.623218 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99f3da-3e06-4502-99ab-6be67c4982a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:26.624822 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99f3da-3e06-4502-99ab-6be67c4982a2 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:26.635047 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1195/2394] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:48:26.639183 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d52d36-4a19-43ae-a3f6-4bb064e6b7b1 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:26.650103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d52d36-4a19-43ae-a3f6-4bb064e6b7b1 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:26.654977 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c4bab-7bd5-4ebe-9084-aa6a150bd7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:26.655791 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c4bab-7bd5-4ebe-9084-aa6a150bd7f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:26.656070 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c4bab-7bd5-4ebe-9084-aa6a150bd7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:26.656416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c4bab-7bd5-4ebe-9084-aa6a150bd7f8 admin admin] PATH_INFO: `/v3/projects/48eeb3fe7e194bf9aea90d74a07ec266/users/b4a8f980f48a4de2a3db6cfbf718beab/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:26.658300 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c4bab-7bd5-4ebe-9084-aa6a150bd7f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48eeb3fe7e194bf9aea90d74a07ec266, user_id=b4a8f980f48a4de2a3db6cfbf718beab, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:26.660972 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c4bab-7bd5-4ebe-9084-aa6a150bd7f8 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:26.671967 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1200/2395] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:48:26 2022] PUT /identity/v3/projects/48eeb3fe7e194bf9aea90d74a07ec266/users/b4a8f980f48a4de2a3db6cfbf718beab/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:48:33.171388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b39f4-0153-4e88-9f3e-eaa125e273fd None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:33.171859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b39f4-0153-4e88-9f3e-eaa125e273fd None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:33.172237 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b39f4-0153-4e88-9f3e-eaa125e273fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:33.192448 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-0c3b39f4-0153-4e88-9f3e-eaa125e273fd None None] MFA Rules not processed for user `d3c1ad9da81f48c390ad82e10e44712d`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:48:33.193810 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-0c3b39f4-0153-4e88-9f3e-eaa125e273fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:33.271249 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1196/2396] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:48:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:48:33.309754 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcf9af-475f-408a-8159-378502581fad None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:33.309754 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcf9af-475f-408a-8159-378502581fad None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:33.309754 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcf9af-475f-408a-8159-378502581fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:33.369913 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-1abcf9af-475f-408a-8159-378502581fad None None] MFA Rules not processed for user `b4a8f980f48a4de2a3db6cfbf718beab`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:48:33.369913 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-1abcf9af-475f-408a-8159-378502581fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:33.415551 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:48:33.475547 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1201/2397] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:48:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:48:33.491557 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b39f4-0153-4e88-9f3e-eaa125e273fd None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:48:33.494194 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-0c3b39f4-0153-4e88-9f3e-eaa125e273fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:33.606843 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b39f4-0153-4e88-9f3e-eaa125e273fd None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:48:33.612625 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33339639-a369-4b81-bd25-b703d21e30be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:48:33.613419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33339639-a369-4b81-bd25-b703d21e30be service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:33.613810 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33339639-a369-4b81-bd25-b703d21e30be service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:33.614192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33339639-a369-4b81-bd25-b703d21e30be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:33.626942 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:48:33.688149 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33339639-a369-4b81-bd25-b703d21e30be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:48:33.690282 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33339639-a369-4b81-bd25-b703d21e30be service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:48:33.715645 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1197/2398] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:48:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:48:37.528994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620845d-47d7-41cf-816d-6e40b406c16b None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:37.529482 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620845d-47d7-41cf-816d-6e40b406c16b None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:37.529563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620845d-47d7-41cf-816d-6e40b406c16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:37.549140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-b620845d-47d7-41cf-816d-6e40b406c16b None None] MFA Rules not processed for user `49ccbcfbd48642aabd74d789b1875dd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:48:37.550477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-b620845d-47d7-41cf-816d-6e40b406c16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:37.564605 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1202/2399] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:48:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:48:42.805169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2afec-fba4-4001-8923-27f1273a16df None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:48:42.806209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2afec-fba4-4001-8923-27f1273a16df None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:48:42.806209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2afec-fba4-4001-8923-27f1273a16df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:48:42.825082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-f7b2afec-fba4-4001-8923-27f1273a16df None None] MFA Rules not processed for user `78b6599200a94a2a8bcc0149dc22944d`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:48:42.826275 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f7b2afec-fba4-4001-8923-27f1273a16df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:48:42.886074 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1198/2400] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:48:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:48:47.543044 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=18 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:48:52.877908 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=47 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:49:08.687587 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c97a2-c843-423f-a8f0-c80f285a9583 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:49:08.688059 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c97a2-c843-423f-a8f0-c80f285a9583 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:49:08.688059 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c97a2-c843-423f-a8f0-c80f285a9583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:49:08.719097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-8e0c97a2-c843-423f-a8f0-c80f285a9583 None None] MFA Rules not processed for user `58e1f1ac6ea14da0be6e9a24d007847a`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:49:08.724082 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-8e0c97a2-c843-423f-a8f0-c80f285a9583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:49:08.797615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1203/2401] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Jun 27 18:49:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:49:08.929568 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2afec-fba4-4001-8923-27f1273a16df None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:49:08.931093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f7b2afec-fba4-4001-8923-27f1273a16df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:49:09.007045 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2afec-fba4-4001-8923-27f1273a16df None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:49:09.012255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d50bd-b4eb-4a64-98f6-9ec81e06a027 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1529c9a45434a19968e924797688f06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:49:09.012897 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d50bd-b4eb-4a64-98f6-9ec81e06a027 service swift] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:49:09.013478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d50bd-b4eb-4a64-98f6-9ec81e06a027 service swift] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:49:09.013478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d50bd-b4eb-4a64-98f6-9ec81e06a027 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:49:09.024072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d50bd-b4eb-4a64-98f6-9ec81e06a027 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:49:09.025637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d50bd-b4eb-4a64-98f6-9ec81e06a027 service swift] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:49:09.033969 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1199/2402] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Jun 27 18:49:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 18:49:18.791014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=25 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:49:19.031848 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=32 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:49:22.003646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9917625-7e52-46ae-81fb-e76de0b26c21 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:49:22.004218 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9917625-7e52-46ae-81fb-e76de0b26c21 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:49:22.004556 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9917625-7e52-46ae-81fb-e76de0b26c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:49:22.036598 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-b9917625-7e52-46ae-81fb-e76de0b26c21 None None] MFA Rules not processed for user `f5f17909688f40f9b752e63b4074c2ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:49:22.038773 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-b9917625-7e52-46ae-81fb-e76de0b26c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:49:22.109727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1204/2403] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:49:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:49:26.697048 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac114fea-637d-4b50-8115-54804f048c8b None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:49:26.697544 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac114fea-637d-4b50-8115-54804f048c8b None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:49:26.697981 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac114fea-637d-4b50-8115-54804f048c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:49:26.724517 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-ac114fea-637d-4b50-8115-54804f048c8b None None] MFA Rules not processed for user `f6d74ace9f674f0da40cb35f39d559c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:49:26.726216 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-ac114fea-637d-4b50-8115-54804f048c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:49:26.807557 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1200/2404] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:49:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:49:31.266489 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a950f08-2a16-4d44-b3c0-f818b92f6ceb None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:49:31.267811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a950f08-2a16-4d44-b3c0-f818b92f6ceb None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:49:31.267811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a950f08-2a16-4d44-b3c0-f818b92f6ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:49:31.288076 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-4a950f08-2a16-4d44-b3c0-f818b92f6ceb None None] MFA Rules not processed for user `58e1f1ac6ea14da0be6e9a24d007847a`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:49:31.289598 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-4a950f08-2a16-4d44-b3c0-f818b92f6ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:49:31.297917 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1205/2405] 10.222.0.15 () {58 vars in 967 bytes} [Mon Jun 27 18:49:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:49:31.312227 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac114fea-637d-4b50-8115-54804f048c8b None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:49:31.325297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac114fea-637d-4b50-8115-54804f048c8b None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:49:31.336653 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b689a-dec7-4815-9987-d25b95e6dc20 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1529c9a45434a19968e924797688f06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:49:31.336653 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b689a-dec7-4815-9987-d25b95e6dc20 service swift] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:49:31.336653 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b689a-dec7-4815-9987-d25b95e6dc20 service swift] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:49:31.336653 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b689a-dec7-4815-9987-d25b95e6dc20 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:49:31.343820 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b689a-dec7-4815-9987-d25b95e6dc20 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:49:31.345497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b689a-dec7-4815-9987-d25b95e6dc20 service swift] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:49:31.355463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1201/2406] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Jun 27 18:49:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 18:49:34.513758 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a950f08-2a16-4d44-b3c0-f818b92f6ceb None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:49:34.515453 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-4a950f08-2a16-4d44-b3c0-f818b92f6ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:49:34.608838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a950f08-2a16-4d44-b3c0-f818b92f6ceb None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:49:34.615144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251e682-08e4-45a1-805e-875b07368ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:49:34.616070 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251e682-08e4-45a1-805e-875b07368ebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:49:34.616351 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251e682-08e4-45a1-805e-875b07368ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:49:34.616622 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251e682-08e4-45a1-805e-875b07368ebb admin admin] PATH_INFO: `/v3/users/d3c1ad9da81f48c390ad82e10e44712d` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:49:34.618328 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1251e682-08e4-45a1-805e-875b07368ebb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3c1ad9da81f48c390ad82e10e44712d)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:49:34.621013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1251e682-08e4-45a1-805e-875b07368ebb admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:49:34.868955 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-1251e682-08e4-45a1-805e-875b07368ebb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3c1ad9da81f48c390ad82e10e44712d'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:49:34.877609 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-1251e682-08e4-45a1-805e-875b07368ebb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3c1ad9da81f48c390ad82e10e44712d'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:49:34.878348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-1251e682-08e4-45a1-805e-875b07368ebb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3c1ad9da81f48c390ad82e10e44712d'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:49:34.879091 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-1251e682-08e4-45a1-805e-875b07368ebb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3c1ad9da81f48c390ad82e10e44712d'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:49:34.899818 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-1251e682-08e4-45a1-805e-875b07368ebb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3c1ad9da81f48c390ad82e10e44712d'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:49:34.977347 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1206/2407] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:49:34 2022] DELETE /identity/v3/users/d3c1ad9da81f48c390ad82e10e44712d => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:49:35.242520 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b689a-dec7-4815-9987-d25b95e6dc20 service swift] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:49:35.253599 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b689a-dec7-4815-9987-d25b95e6dc20 service swift] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:49:35.311687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acac07a-00d4-41af-b081-6d2038e23f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:49:35.312671 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acac07a-00d4-41af-b081-6d2038e23f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:49:35.312955 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acac07a-00d4-41af-b081-6d2038e23f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:49:35.313261 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acac07a-00d4-41af-b081-6d2038e23f4f admin admin] PATH_INFO: `/v3/projects/bc7a3d93a3a84224b7b23d93924f276f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:49:35.314498 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acac07a-00d4-41af-b081-6d2038e23f4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc7a3d93a3a84224b7b23d93924f276f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:49:35.316468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acac07a-00d4-41af-b081-6d2038e23f4f admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:49:35.423788 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-4acac07a-00d4-41af-b081-6d2038e23f4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc7a3d93a3a84224b7b23d93924f276f', 'request_id': 'req-4acac07a-00d4-41af-b081-6d2038e23f4f'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:49:35.520447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-4acac07a-00d4-41af-b081-6d2038e23f4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc7a3d93a3a84224b7b23d93924f276f', 'request_id': 'req-4acac07a-00d4-41af-b081-6d2038e23f4f'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:49:35.527247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-4acac07a-00d4-41af-b081-6d2038e23f4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc7a3d93a3a84224b7b23d93924f276f', 'request_id': 'req-4acac07a-00d4-41af-b081-6d2038e23f4f'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:49:35.538603 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-4acac07a-00d4-41af-b081-6d2038e23f4f admin admin] The token cache is being invalidate because project bc7a3d93a3a84224b7b23d93924f276f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:49:35.538892 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-4acac07a-00d4-41af-b081-6d2038e23f4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:49:35.539522 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-4acac07a-00d4-41af-b081-6d2038e23f4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:49:35.541098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1202/2408] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:49:35 2022] DELETE /identity/v3/projects/bc7a3d93a3a84224b7b23d93924f276f => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:49:39.602551 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251e682-08e4-45a1-805e-875b07368ebb admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:49:39.603971 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-1251e682-08e4-45a1-805e-875b07368ebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:49:39.627116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:49:39.664555 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251e682-08e4-45a1-805e-875b07368ebb admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:49:39.717026 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659bdfb-5221-4c2a-888a-a741bf4683f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:49:39.717600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659bdfb-5221-4c2a-888a-a741bf4683f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:49:39.717873 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659bdfb-5221-4c2a-888a-a741bf4683f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:49:39.718147 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659bdfb-5221-4c2a-888a-a741bf4683f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:49:39.721054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-b659bdfb-5221-4c2a-888a-a741bf4683f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:49:39.793850 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659bdfb-5221-4c2a-888a-a741bf4683f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:49:39.795328 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659bdfb-5221-4c2a-888a-a741bf4683f6 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:49:39.806519 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1207/2409] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:49:39 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:49:45.535509 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:49:49.802316 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=62 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:50:03.935231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acac07a-00d4-41af-b081-6d2038e23f4f admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:50:03.936671 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-4acac07a-00d4-41af-b081-6d2038e23f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:50:03.950831 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acac07a-00d4-41af-b081-6d2038e23f4f admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:50:03.956289 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc7b2f-1bb6-4588-922c-002695ce18c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:50:03.956289 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc7b2f-1bb6-4588-922c-002695ce18c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:50:03.956289 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc7b2f-1bb6-4588-922c-002695ce18c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:50:03.956289 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc7b2f-1bb6-4588-922c-002695ce18c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:50:03.957104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-afdc7b2f-1bb6-4588-922c-002695ce18c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:50:04.034984 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc7b2f-1bb6-4588-922c-002695ce18c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:50:04.036435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdc7b2f-1bb6-4588-922c-002695ce18c8 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:50:04.050459 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1203/2410] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:50:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:50:05.178823 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659bdfb-5221-4c2a-888a-a741bf4683f6 service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:50:05.189797 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659bdfb-5221-4c2a-888a-a741bf4683f6 service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:50:05.194072 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf078970-3cfe-434d-ab31-161b5adf0ea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:50:05.194990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf078970-3cfe-434d-ab31-161b5adf0ea4 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:50:05.195276 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf078970-3cfe-434d-ab31-161b5adf0ea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:50:05.195655 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf078970-3cfe-434d-ab31-161b5adf0ea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:50:05.197539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-bf078970-3cfe-434d-ab31-161b5adf0ea4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:50:05.213370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf078970-3cfe-434d-ab31-161b5adf0ea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:50:05.214880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf078970-3cfe-434d-ab31-161b5adf0ea4 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:50:05.229964 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1208/2411] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:50:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:50:14.044320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=34 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:50:14.065729 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc7b2f-1bb6-4588-922c-002695ce18c8 service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:50:14.067144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-afdc7b2f-1bb6-4588-922c-002695ce18c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:50:14.127989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdc7b2f-1bb6-4588-922c-002695ce18c8 service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:50:14.131722 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0a8a7-d5af-45da-ac6a-292dc8eb19fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:50:14.132923 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0a8a7-d5af-45da-ac6a-292dc8eb19fe service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:50:14.133282 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0a8a7-d5af-45da-ac6a-292dc8eb19fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:50:14.133550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0a8a7-d5af-45da-ac6a-292dc8eb19fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:50:14.135326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-8ed0a8a7-d5af-45da-ac6a-292dc8eb19fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:50:14.193428 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0a8a7-d5af-45da-ac6a-292dc8eb19fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:50:14.195599 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0a8a7-d5af-45da-ac6a-292dc8eb19fe service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:50:14.204661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1204/2412] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:50:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:50:15.222506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=6 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:50:22.740605 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca9068-16be-4327-b41c-072ca3296ee6 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:50:22.741262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca9068-16be-4327-b41c-072ca3296ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:50:22.741262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca9068-16be-4327-b41c-072ca3296ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:50:22.759584 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-90ca9068-16be-4327-b41c-072ca3296ee6 None None] MFA Rules not processed for user `2fc6711e48be4d4ab33695e06c7227ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:50:22.760770 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-90ca9068-16be-4327-b41c-072ca3296ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:50:22.767640 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1209/2413] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:50:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:50:23.150182 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0a8a7-d5af-45da-ac6a-292dc8eb19fe service cinder] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:50:23.159617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0a8a7-d5af-45da-ac6a-292dc8eb19fe service cinder] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:50:23.163217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b211ba2-d9cf-4be2-9932-e805394f4c79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:50:23.163880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b211ba2-d9cf-4be2-9932-e805394f4c79 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:50:23.164190 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b211ba2-d9cf-4be2-9932-e805394f4c79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:50:23.164466 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b211ba2-d9cf-4be2-9932-e805394f4c79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:50:23.166252 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-1b211ba2-d9cf-4be2-9932-e805394f4c79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:50:23.175584 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b211ba2-d9cf-4be2-9932-e805394f4c79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:50:23.176989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b211ba2-d9cf-4be2-9932-e805394f4c79 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:50:23.184951 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1205/2414] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:50:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:50:23.578880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca9068-16be-4327-b41c-072ca3296ee6 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:50:23.580561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-90ca9068-16be-4327-b41c-072ca3296ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:50:23.658261 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca9068-16be-4327-b41c-072ca3296ee6 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:50:23.663277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4000dfa-f02c-4013-a970-f3138d16220d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cd6f0a1286243e4add692db01028205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:50:23.663992 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4000dfa-f02c-4013-a970-f3138d16220d service placement] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:50:23.664470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4000dfa-f02c-4013-a970-f3138d16220d service placement] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:50:23.664904 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4000dfa-f02c-4013-a970-f3138d16220d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:50:23.666718 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-a4000dfa-f02c-4013-a970-f3138d16220d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:50:23.678331 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4000dfa-f02c-4013-a970-f3138d16220d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:50:23.680806 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4000dfa-f02c-4013-a970-f3138d16220d service placement] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:50:23.691416 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1210/2415] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:50:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:50:26.718681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b629b3f-775b-4b46-ac1a-f56480731e04 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:50:26.719176 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b629b3f-775b-4b46-ac1a-f56480731e04 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:50:26.719419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b629b3f-775b-4b46-ac1a-f56480731e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:50:26.738278 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-9b629b3f-775b-4b46-ac1a-f56480731e04 None None] MFA Rules not processed for user `78b6599200a94a2a8bcc0149dc22944d`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:50:26.739816 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-9b629b3f-775b-4b46-ac1a-f56480731e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:50:26.797430 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1206/2416] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:50:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:50:29.951744 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4000dfa-f02c-4013-a970-f3138d16220d service placement] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:50:29.961344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4000dfa-f02c-4013-a970-f3138d16220d service placement] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:50:29.965434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c97e43-bc29-4f8b-870a-45563c04b7fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:50:29.966256 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c97e43-bc29-4f8b-870a-45563c04b7fd service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:50:29.966581 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c97e43-bc29-4f8b-870a-45563c04b7fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:50:29.966862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c97e43-bc29-4f8b-870a-45563c04b7fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:50:29.968677 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-48c97e43-bc29-4f8b-870a-45563c04b7fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:50:29.978877 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c97e43-bc29-4f8b-870a-45563c04b7fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:50:29.981910 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c97e43-bc29-4f8b-870a-45563c04b7fd service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:50:29.989299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1211/2417] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:50:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:50:33.959686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b629b3f-775b-4b46-ac1a-f56480731e04 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:50:33.961082 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-9b629b3f-775b-4b46-ac1a-f56480731e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:50:33.977478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b629b3f-775b-4b46-ac1a-f56480731e04 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:50:33.983244 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecd7cb-c405-46ca-a4c1-43fdd17588e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:50:33.983879 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecd7cb-c405-46ca-a4c1-43fdd17588e1 service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:50:33.984159 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecd7cb-c405-46ca-a4c1-43fdd17588e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:50:33.984434 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecd7cb-c405-46ca-a4c1-43fdd17588e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:50:33.986140 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-1cecd7cb-c405-46ca-a4c1-43fdd17588e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:50:33.999288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cecd7cb-c405-46ca-a4c1-43fdd17588e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:50:34.000985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cecd7cb-c405-46ca-a4c1-43fdd17588e1 service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:50:34.011789 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1207/2418] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:50:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:50:35.379768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c97e43-bc29-4f8b-870a-45563c04b7fd service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:50:35.393506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c97e43-bc29-4f8b-870a-45563c04b7fd service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:50:35.399984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b73664-5f17-4f4e-b5fc-0adb0dfe2d43 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cd6f0a1286243e4add692db01028205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:50:35.400624 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b73664-5f17-4f4e-b5fc-0adb0dfe2d43 service placement] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:50:35.400900 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b73664-5f17-4f4e-b5fc-0adb0dfe2d43 service placement] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:50:35.401173 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b73664-5f17-4f4e-b5fc-0adb0dfe2d43 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:50:35.402971 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-67b73664-5f17-4f4e-b5fc-0adb0dfe2d43 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:50:35.415405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b73664-5f17-4f4e-b5fc-0adb0dfe2d43 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:50:35.416987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b73664-5f17-4f4e-b5fc-0adb0dfe2d43 service placement] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:50:35.428479 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1212/2419] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:50:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:50:35.508274 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecd7cb-c405-46ca-a4c1-43fdd17588e1 service nova] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:50:35.510773 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-1cecd7cb-c405-46ca-a4c1-43fdd17588e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:50:35.568454 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:50:35.609622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecd7cb-c405-46ca-a4c1-43fdd17588e1 service nova] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:50:35.617602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6caca6-e462-41f5-ad23-318f9c179071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:50:35.618705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6caca6-e462-41f5-ad23-318f9c179071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:50:35.619222 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6caca6-e462-41f5-ad23-318f9c179071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:50:35.619669 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6caca6-e462-41f5-ad23-318f9c179071 admin admin] PATH_INFO: `/v3/users/f5f17909688f40f9b752e63b4074c2ee` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:50:35.620901 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6caca6-e462-41f5-ad23-318f9c179071 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5f17909688f40f9b752e63b4074c2ee)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:50:35.623002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6caca6-e462-41f5-ad23-318f9c179071 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:50:35.855671 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-1f6caca6-e462-41f5-ad23-318f9c179071 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5f17909688f40f9b752e63b4074c2ee'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:50:35.863630 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-1f6caca6-e462-41f5-ad23-318f9c179071 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5f17909688f40f9b752e63b4074c2ee'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:50:35.864312 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-1f6caca6-e462-41f5-ad23-318f9c179071 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5f17909688f40f9b752e63b4074c2ee'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:50:35.877348 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-1f6caca6-e462-41f5-ad23-318f9c179071 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5f17909688f40f9b752e63b4074c2ee'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:50:35.956464 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-1f6caca6-e462-41f5-ad23-318f9c179071 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5f17909688f40f9b752e63b4074c2ee'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:50:35.963111 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1208/2420] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:50:35 2022] DELETE /identity/v3/users/f5f17909688f40f9b752e63b4074c2ee => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:50:36.189684 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b73664-5f17-4f4e-b5fc-0adb0dfe2d43 service placement] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:50:36.200839 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b73664-5f17-4f4e-b5fc-0adb0dfe2d43 service placement] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:50:36.258689 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915b4ba-de7a-4031-9bbd-49f37e265feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:50:36.259469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915b4ba-de7a-4031-9bbd-49f37e265feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:50:36.259750 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915b4ba-de7a-4031-9bbd-49f37e265feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:50:36.260023 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915b4ba-de7a-4031-9bbd-49f37e265feb admin admin] PATH_INFO: `/v3/projects/5284ec5970e14418bfbd5391ee3b282c` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:50:36.261187 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915b4ba-de7a-4031-9bbd-49f37e265feb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5284ec5970e14418bfbd5391ee3b282c)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:50:36.263434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915b4ba-de7a-4031-9bbd-49f37e265feb admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:50:36.437335 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-1915b4ba-de7a-4031-9bbd-49f37e265feb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5284ec5970e14418bfbd5391ee3b282c', 'request_id': 'req-1915b4ba-de7a-4031-9bbd-49f37e265feb'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:50:36.567435 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-1915b4ba-de7a-4031-9bbd-49f37e265feb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5284ec5970e14418bfbd5391ee3b282c', 'request_id': 'req-1915b4ba-de7a-4031-9bbd-49f37e265feb'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:50:36.569621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:50:36.574575 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-1915b4ba-de7a-4031-9bbd-49f37e265feb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5284ec5970e14418bfbd5391ee3b282c', 'request_id': 'req-1915b4ba-de7a-4031-9bbd-49f37e265feb'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:50:36.585249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-1915b4ba-de7a-4031-9bbd-49f37e265feb admin admin] The token cache is being invalidate because project 5284ec5970e14418bfbd5391ee3b282c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:50:36.585537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-1915b4ba-de7a-4031-9bbd-49f37e265feb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:50:36.586174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-1915b4ba-de7a-4031-9bbd-49f37e265feb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:50:36.587730 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1213/2421] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:50:36 2022] DELETE /identity/v3/projects/5284ec5970e14418bfbd5391ee3b282c => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:50:39.044774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6caca6-e462-41f5-ad23-318f9c179071 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:50:39.046979 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-1f6caca6-e462-41f5-ad23-318f9c179071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:50:39.121892 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6caca6-e462-41f5-ad23-318f9c179071 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:50:39.167915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:50:39.184331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96504ee-0f05-45f3-800a-55d8b4773689 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:50:39.185068 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96504ee-0f05-45f3-800a-55d8b4773689 service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:50:39.185358 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96504ee-0f05-45f3-800a-55d8b4773689 service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:50:39.185669 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96504ee-0f05-45f3-800a-55d8b4773689 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:50:39.188110 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-b96504ee-0f05-45f3-800a-55d8b4773689 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:50:39.200841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96504ee-0f05-45f3-800a-55d8b4773689 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:50:39.202681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96504ee-0f05-45f3-800a-55d8b4773689 service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:50:39.213419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1209/2422] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:50:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:50:39.448116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915b4ba-de7a-4031-9bbd-49f37e265feb admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:50:39.450268 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-1915b4ba-de7a-4031-9bbd-49f37e265feb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:50:39.474713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915b4ba-de7a-4031-9bbd-49f37e265feb admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:50:39.481683 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8625fc7-d5f8-4e4b-921a-d9e42b464338 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:50:39.482526 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8625fc7-d5f8-4e4b-921a-d9e42b464338 service nova] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:50:39.482999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8625fc7-d5f8-4e4b-921a-d9e42b464338 service nova] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:50:39.483563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8625fc7-d5f8-4e4b-921a-d9e42b464338 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:50:39.486271 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-f8625fc7-d5f8-4e4b-921a-d9e42b464338 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:50:39.564677 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8625fc7-d5f8-4e4b-921a-d9e42b464338 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:50:39.567054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8625fc7-d5f8-4e4b-921a-d9e42b464338 service nova] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:50:39.579225 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1214/2423] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:50:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:50:49.247957 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=11 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:50:49.574077 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=32,keystone:DELETE=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:50:55.845561 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89071a2e-e7cc-43e0-b581-3313a7197207 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:50:55.846106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89071a2e-e7cc-43e0-b581-3313a7197207 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:50:55.846344 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89071a2e-e7cc-43e0-b581-3313a7197207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:50:55.865810 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-89071a2e-e7cc-43e0-b581-3313a7197207 None None] MFA Rules not processed for user `58e1f1ac6ea14da0be6e9a24d007847a`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:50:55.867051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-89071a2e-e7cc-43e0-b581-3313a7197207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:50:55.925530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1210/2424] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Jun 27 18:50:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:50:56.069272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8625fc7-d5f8-4e4b-921a-d9e42b464338 service nova] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:50:56.070721 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-f8625fc7-d5f8-4e4b-921a-d9e42b464338 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:50:56.139659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8625fc7-d5f8-4e4b-921a-d9e42b464338 service nova] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:50:56.144733 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7cf79-266b-4564-a186-d8d41eea18ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1529c9a45434a19968e924797688f06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:50:56.145378 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7cf79-266b-4564-a186-d8d41eea18ac service swift] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:50:56.145662 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7cf79-266b-4564-a186-d8d41eea18ac service swift] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:50:56.145938 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7cf79-266b-4564-a186-d8d41eea18ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:50:56.153996 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7cf79-266b-4564-a186-d8d41eea18ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:50:56.155648 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7cf79-266b-4564-a186-d8d41eea18ac service swift] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:50:56.163811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1215/2425] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Jun 27 18:50:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 18:51:05.917540 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=25 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:51:06.160562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=32 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:51:09.532478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89071a2e-e7cc-43e0-b581-3313a7197207 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:09.547141 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89071a2e-e7cc-43e0-b581-3313a7197207 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:09.553249 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bd799-f46c-40bb-ac4f-72cfc2a9690b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:09.553927 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bd799-f46c-40bb-ac4f-72cfc2a9690b service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:09.554181 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bd799-f46c-40bb-ac4f-72cfc2a9690b service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:09.554576 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bd799-f46c-40bb-ac4f-72cfc2a9690b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:09.556411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-542bd799-f46c-40bb-ac4f-72cfc2a9690b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:09.570364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bd799-f46c-40bb-ac4f-72cfc2a9690b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:09.571950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bd799-f46c-40bb-ac4f-72cfc2a9690b service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:09.583001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1211/2426] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:51:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:51:11.542903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac17e0-0bf7-4700-8238-abe3115611cd None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:11.543449 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac17e0-0bf7-4700-8238-abe3115611cd None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:11.543449 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac17e0-0bf7-4700-8238-abe3115611cd None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:11.545185 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1216/2427] 10.222.0.15 () {54 vars in 921 bytes} [Mon Jun 27 18:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:51:11.551511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41ce7f-5e9f-4ee5-88e8-84420f75f7b4 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:11.552530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41ce7f-5e9f-4ee5-88e8-84420f75f7b4 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:11.552992 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41ce7f-5e9f-4ee5-88e8-84420f75f7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:11.571700 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-0c41ce7f-5e9f-4ee5-88e8-84420f75f7b4 None None] MFA Rules not processed for user `1dddd4efb44c4152bb4151f7715f2223`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:51:11.572906 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-0c41ce7f-5e9f-4ee5-88e8-84420f75f7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:11.583828 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1212/2428] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Jun 27 18:51:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:51:11.596306 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac17e0-0bf7-4700-8238-abe3115611cd None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:11.598578 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-7dac17e0-0bf7-4700-8238-abe3115611cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:11.680968 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac17e0-0bf7-4700-8238-abe3115611cd None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:11.684512 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623552f0-ea83-49f1-a4e3-0dc5d154f056 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e75e5ae3ca45d0b503f6e52f399558', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:11.685152 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623552f0-ea83-49f1-a4e3-0dc5d154f056 service glance] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:11.685435 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623552f0-ea83-49f1-a4e3-0dc5d154f056 service glance] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:11.685713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623552f0-ea83-49f1-a4e3-0dc5d154f056 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:11.698209 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623552f0-ea83-49f1-a4e3-0dc5d154f056 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:11.700007 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623552f0-ea83-49f1-a4e3-0dc5d154f056 service glance] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:11.722090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1217/2429] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:51:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:51:11.883647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e382a6-7ec4-40c9-a981-84ff444b80b5 None None] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:11.883935 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e382a6-7ec4-40c9-a981-84ff444b80b5 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:11.884214 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e382a6-7ec4-40c9-a981-84ff444b80b5 None None] PATH_INFO: `/` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:11.885636 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1213/2430] 10.222.0.15 () {54 vars in 921 bytes} [Mon Jun 27 18:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:51:11.890349 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94211d-8b74-4f83-982a-5a7759c5d931 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:11.890683 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94211d-8b74-4f83-982a-5a7759c5d931 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:11.890969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94211d-8b74-4f83-982a-5a7759c5d931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:11.910234 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-6e94211d-8b74-4f83-982a-5a7759c5d931 None None] MFA Rules not processed for user `dbe621916dbe4f1a8686097fe2bdf762`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:51:11.911465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-6e94211d-8b74-4f83-982a-5a7759c5d931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:11.961413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1218/2431] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Jun 27 18:51:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:51:11.973075 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e382a6-7ec4-40c9-a981-84ff444b80b5 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:11.974749 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-42e382a6-7ec4-40c9-a981-84ff444b80b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:11.987609 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e382a6-7ec4-40c9-a981-84ff444b80b5 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:11.991304 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4bcfce-f62e-41bd-b1b5-bd93a8d33e76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:11.991924 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bcfce-f62e-41bd-b1b5-bd93a8d33e76 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:11.992204 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bcfce-f62e-41bd-b1b5-bd93a8d33e76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:11.992479 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bcfce-f62e-41bd-b1b5-bd93a8d33e76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:11.999350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4bcfce-f62e-41bd-b1b5-bd93a8d33e76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:12.000764 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4bcfce-f62e-41bd-b1b5-bd93a8d33e76 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:12.009040 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1214/2432] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:51:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:51:12.092401 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389d9d5-85cc-4394-a06a-e27d79b2d899 None None] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:12.092688 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389d9d5-85cc-4394-a06a-e27d79b2d899 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:12.092969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389d9d5-85cc-4394-a06a-e27d79b2d899 None None] PATH_INFO: `/` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:12.094342 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1219/2433] 10.222.0.15 () {54 vars in 921 bytes} [Mon Jun 27 18:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 18:51:12.100186 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452c9a9-4fb4-4b9f-932b-b232db8dda6b None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:12.100470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452c9a9-4fb4-4b9f-932b-b232db8dda6b None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:12.100746 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452c9a9-4fb4-4b9f-932b-b232db8dda6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:12.118937 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-2452c9a9-4fb4-4b9f-932b-b232db8dda6b None None] MFA Rules not processed for user `1dddd4efb44c4152bb4151f7715f2223`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:51:12.120144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-2452c9a9-4fb4-4b9f-932b-b232db8dda6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:12.129702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1215/2434] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Jun 27 18:51:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:51:12.139771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389d9d5-85cc-4394-a06a-e27d79b2d899 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:12.151461 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389d9d5-85cc-4394-a06a-e27d79b2d899 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:12.154961 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20801f04-54ca-4a47-b1c5-b7aca0856231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:12.155583 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20801f04-54ca-4a47-b1c5-b7aca0856231 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:12.155866 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20801f04-54ca-4a47-b1c5-b7aca0856231 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:12.156142 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20801f04-54ca-4a47-b1c5-b7aca0856231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:12.166376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20801f04-54ca-4a47-b1c5-b7aca0856231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:12.167822 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20801f04-54ca-4a47-b1c5-b7aca0856231 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:12.178343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1220/2435] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:51:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:51:16.594345 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d55cc3-630e-4d9d-8c9c-2be1d91c4d8e None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:16.594833 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d55cc3-630e-4d9d-8c9c-2be1d91c4d8e None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:16.595020 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d55cc3-630e-4d9d-8c9c-2be1d91c4d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:16.619743 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-01d55cc3-630e-4d9d-8c9c-2be1d91c4d8e None None] MFA Rules not processed for user `58e1f1ac6ea14da0be6e9a24d007847a`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:51:16.621363 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-01d55cc3-630e-4d9d-8c9c-2be1d91c4d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:16.637434 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1216/2436] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Jun 27 18:51:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:51:16.647192 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20801f04-54ca-4a47-b1c5-b7aca0856231 service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:16.660473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20801f04-54ca-4a47-b1c5-b7aca0856231 service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:16.668411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dfce47-9c5c-4a85-8da4-5d4849d8c3d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1529c9a45434a19968e924797688f06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:16.669361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dfce47-9c5c-4a85-8da4-5d4849d8c3d9 service swift] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:16.669845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dfce47-9c5c-4a85-8da4-5d4849d8c3d9 service swift] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:16.670289 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dfce47-9c5c-4a85-8da4-5d4849d8c3d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:16.681220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dfce47-9c5c-4a85-8da4-5d4849d8c3d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:16.683279 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dfce47-9c5c-4a85-8da4-5d4849d8c3d9 service swift] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:16.693075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1221/2437] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Jun 27 18:51:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 18:51:26.609911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=21 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:51:26.688813 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=70 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:51:33.943422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab8e88-f257-43fa-ba0f-b183caf4d1ad None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:33.943904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab8e88-f257-43fa-ba0f-b183caf4d1ad None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:33.944098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab8e88-f257-43fa-ba0f-b183caf4d1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:33.964156 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-1bab8e88-f257-43fa-ba0f-b183caf4d1ad None None] MFA Rules not processed for user `58e1f1ac6ea14da0be6e9a24d007847a`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:51:33.965260 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-1bab8e88-f257-43fa-ba0f-b183caf4d1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:33.972519 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1217/2438] 10.222.0.15 () {58 vars in 967 bytes} [Mon Jun 27 18:51:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:51:33.984337 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dfce47-9c5c-4a85-8da4-5d4849d8c3d9 service swift] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:33.995583 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dfce47-9c5c-4a85-8da4-5d4849d8c3d9 service swift] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:34.000584 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a53706-b1da-47c9-974b-72f91dd95b9f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1529c9a45434a19968e924797688f06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:34.001202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a53706-b1da-47c9-974b-72f91dd95b9f service swift] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:34.001484 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a53706-b1da-47c9-974b-72f91dd95b9f service swift] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:34.001756 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a53706-b1da-47c9-974b-72f91dd95b9f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:34.010474 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a53706-b1da-47c9-974b-72f91dd95b9f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:34.012039 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a53706-b1da-47c9-974b-72f91dd95b9f service swift] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:34.020112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1222/2439] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Jun 27 18:51:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 18:51:40.152796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab8e88-f257-43fa-ba0f-b183caf4d1ad None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:40.154267 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-1bab8e88-f257-43fa-ba0f-b183caf4d1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:40.221480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab8e88-f257-43fa-ba0f-b183caf4d1ad None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:40.226705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc2bc5-2106-42c3-a752-1648f4c60cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:40.227583 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc2bc5-2106-42c3-a752-1648f4c60cce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:40.227867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc2bc5-2106-42c3-a752-1648f4c60cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:40.228145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc2bc5-2106-42c3-a752-1648f4c60cce admin admin] PATH_INFO: `/v3/users/78b6599200a94a2a8bcc0149dc22944d` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:40.229352 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc2bc5-2106-42c3-a752-1648f4c60cce admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b6599200a94a2a8bcc0149dc22944d)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:40.231455 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc2bc5-2106-42c3-a752-1648f4c60cce admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:40.427571 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-30cc2bc5-2106-42c3-a752-1648f4c60cce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b6599200a94a2a8bcc0149dc22944d'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:40.434482 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-30cc2bc5-2106-42c3-a752-1648f4c60cce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b6599200a94a2a8bcc0149dc22944d'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:40.435138 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-30cc2bc5-2106-42c3-a752-1648f4c60cce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b6599200a94a2a8bcc0149dc22944d'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:40.446763 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-30cc2bc5-2106-42c3-a752-1648f4c60cce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b6599200a94a2a8bcc0149dc22944d'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:40.522571 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-30cc2bc5-2106-42c3-a752-1648f4c60cce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b6599200a94a2a8bcc0149dc22944d'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:40.528765 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1218/2440] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:51:40 2022] DELETE /identity/v3/users/78b6599200a94a2a8bcc0149dc22944d => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:51:40.793528 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a53706-b1da-47c9-974b-72f91dd95b9f service swift] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:40.805771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a53706-b1da-47c9-974b-72f91dd95b9f service swift] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:40.872627 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3668586-a799-4ff4-bbe8-5444e0511948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:40.876731 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3668586-a799-4ff4-bbe8-5444e0511948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:40.876731 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3668586-a799-4ff4-bbe8-5444e0511948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:40.876731 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3668586-a799-4ff4-bbe8-5444e0511948 admin admin] PATH_INFO: `/v3/projects/e03ab1e3fbe7445da087c631d8af557b` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:40.876731 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3668586-a799-4ff4-bbe8-5444e0511948 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e03ab1e3fbe7445da087c631d8af557b)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:40.877300 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3668586-a799-4ff4-bbe8-5444e0511948 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:40.974785 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f3668586-a799-4ff4-bbe8-5444e0511948 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e03ab1e3fbe7445da087c631d8af557b', 'request_id': 'req-f3668586-a799-4ff4-bbe8-5444e0511948'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:41.066746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f3668586-a799-4ff4-bbe8-5444e0511948 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e03ab1e3fbe7445da087c631d8af557b', 'request_id': 'req-f3668586-a799-4ff4-bbe8-5444e0511948'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:41.073507 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f3668586-a799-4ff4-bbe8-5444e0511948 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e03ab1e3fbe7445da087c631d8af557b', 'request_id': 'req-f3668586-a799-4ff4-bbe8-5444e0511948'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:41.084705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f3668586-a799-4ff4-bbe8-5444e0511948 admin admin] The token cache is being invalidate because project e03ab1e3fbe7445da087c631d8af557b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:51:41.085012 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f3668586-a799-4ff4-bbe8-5444e0511948 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:41.085655 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f3668586-a799-4ff4-bbe8-5444e0511948 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:41.087367 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1223/2441] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:51:40 2022] DELETE /identity/v3/projects/e03ab1e3fbe7445da087c631d8af557b => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:51:41.092230 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1784d54-1ab8-4a20-9c0c-7bd9f2c4fdce None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:41.092519 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1784d54-1ab8-4a20-9c0c-7bd9f2c4fdce None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:41.092804 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1784d54-1ab8-4a20-9c0c-7bd9f2c4fdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:41.111050 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-d1784d54-1ab8-4a20-9c0c-7bd9f2c4fdce None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:51:41.112325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-d1784d54-1ab8-4a20-9c0c-7bd9f2c4fdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:41.171549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1219/2442] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:51:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:51:41.186140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a351504-44bc-40b8-bea7-b6dba299eaf3 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:41.186433 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a351504-44bc-40b8-bea7-b6dba299eaf3 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:41.186719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a351504-44bc-40b8-bea7-b6dba299eaf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:41.205547 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-9a351504-44bc-40b8-bea7-b6dba299eaf3 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:51:41.206843 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-9a351504-44bc-40b8-bea7-b6dba299eaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:41.214761 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1224/2443] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:51:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:51:41.218988 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1784d54-1ab8-4a20-9c0c-7bd9f2c4fdce None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:41.228814 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1784d54-1ab8-4a20-9c0c-7bd9f2c4fdce None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:41.233437 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949ddce-38d9-4436-bb0f-e6a6c3c44773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:41.234086 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949ddce-38d9-4436-bb0f-e6a6c3c44773 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:41.234372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949ddce-38d9-4436-bb0f-e6a6c3c44773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:41.234682 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949ddce-38d9-4436-bb0f-e6a6c3c44773 admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:41.235409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949ddce-38d9-4436-bb0f-e6a6c3c44773 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:51:41.235718 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949ddce-38d9-4436-bb0f-e6a6c3c44773 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:41.238212 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949ddce-38d9-4436-bb0f-e6a6c3c44773 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:41.246740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:51:41.249955 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1220/2444] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:51:41.259512 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a351504-44bc-40b8-bea7-b6dba299eaf3 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:41.269004 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a351504-44bc-40b8-bea7-b6dba299eaf3 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:41.273862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a4115-6b5a-4a80-981b-9ec94eca9673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:41.274546 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a4115-6b5a-4a80-981b-9ec94eca9673 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:41.274831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a4115-6b5a-4a80-981b-9ec94eca9673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:41.275108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a4115-6b5a-4a80-981b-9ec94eca9673 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:41.275796 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a4115-6b5a-4a80-981b-9ec94eca9673 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:41.277705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a4115-6b5a-4a80-981b-9ec94eca9673 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:41.371925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1225/2445] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:51:41 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:51:41.376053 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949ddce-38d9-4436-bb0f-e6a6c3c44773 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:41.387473 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949ddce-38d9-4436-bb0f-e6a6c3c44773 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:41.446613 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ef126-67f9-4623-b410-11355b6f2ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:41.447280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ef126-67f9-4623-b410-11355b6f2ee5 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:41.447577 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ef126-67f9-4623-b410-11355b6f2ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:41.447891 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ef126-67f9-4623-b410-11355b6f2ee5 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:41.448593 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ef126-67f9-4623-b410-11355b6f2ee5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:41.450539 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ef126-67f9-4623-b410-11355b6f2ee5 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:41.517408 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1221/2446] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:51:41 2022] POST /identity/v3/users => generated 291 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:51:41.521800 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a4115-6b5a-4a80-981b-9ec94eca9673 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:41.532429 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a4115-6b5a-4a80-981b-9ec94eca9673 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:41.538102 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166f883-399b-47fc-be9e-017cf03802ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:41.538838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166f883-399b-47fc-be9e-017cf03802ec admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:41.539191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166f883-399b-47fc-be9e-017cf03802ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:41.539505 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166f883-399b-47fc-be9e-017cf03802ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:41.540392 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9166f883-399b-47fc-be9e-017cf03802ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:41.542203 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9166f883-399b-47fc-be9e-017cf03802ec admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:41.553245 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1226/2447] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:51:41.557230 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ef126-67f9-4623-b410-11355b6f2ee5 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:41.567212 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ef126-67f9-4623-b410-11355b6f2ee5 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:41.571837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8196f8-e5c2-4cc8-9431-75f1c69d08af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:41.572477 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8196f8-e5c2-4cc8-9431-75f1c69d08af admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:41.577547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8196f8-e5c2-4cc8-9431-75f1c69d08af admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:41.577547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8196f8-e5c2-4cc8-9431-75f1c69d08af admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:41.577547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8196f8-e5c2-4cc8-9431-75f1c69d08af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:41.577547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8196f8-e5c2-4cc8-9431-75f1c69d08af admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:41.586298 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1222/2448] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:51:41.590323 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166f883-399b-47fc-be9e-017cf03802ec admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:41.601332 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166f883-399b-47fc-be9e-017cf03802ec admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:41.605932 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0880cb-d63d-4a55-877c-59d665aa3f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:41.606827 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0880cb-d63d-4a55-877c-59d665aa3f0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:41.607111 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0880cb-d63d-4a55-877c-59d665aa3f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:41.607388 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0880cb-d63d-4a55-877c-59d665aa3f0a admin admin] PATH_INFO: `/v3/projects/1e8490a3f9ad48678c32d42cf10f1763/users/bc24ddc08f9f4466b43c35430b48b467/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:41.622903 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0880cb-d63d-4a55-877c-59d665aa3f0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e8490a3f9ad48678c32d42cf10f1763, user_id=bc24ddc08f9f4466b43c35430b48b467, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:41.625502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0880cb-d63d-4a55-877c-59d665aa3f0a admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:41.688773 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1227/2449] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:51:41 2022] PUT /identity/v3/projects/1e8490a3f9ad48678c32d42cf10f1763/users/bc24ddc08f9f4466b43c35430b48b467/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:51:41.699959 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792444c-ecf5-4e03-a175-29af4c1c58de None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:41.700273 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792444c-ecf5-4e03-a175-29af4c1c58de None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:41.700553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792444c-ecf5-4e03-a175-29af4c1c58de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:41.727137 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-e792444c-ecf5-4e03-a175-29af4c1c58de None None] MFA Rules not processed for user `bc24ddc08f9f4466b43c35430b48b467`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:51:41.728381 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-e792444c-ecf5-4e03-a175-29af4c1c58de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:41.792461 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1223/2450] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:51:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:51:41.796914 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0880cb-d63d-4a55-877c-59d665aa3f0a admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:41.803179 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:51:41.857976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0880cb-d63d-4a55-877c-59d665aa3f0a admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:41.863724 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea44985-c524-4963-b894-5533d392a3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:41.864375 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea44985-c524-4963-b894-5533d392a3fd admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:41.864659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea44985-c524-4963-b894-5533d392a3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:41.864939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea44985-c524-4963-b894-5533d392a3fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:41.865883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea44985-c524-4963-b894-5533d392a3fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:41.867857 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea44985-c524-4963-b894-5533d392a3fd admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:41.930172 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1228/2451] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:51:41 2022] POST /identity/v3/projects => generated 379 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:51:41.933987 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792444c-ecf5-4e03-a175-29af4c1c58de None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:41.945378 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792444c-ecf5-4e03-a175-29af4c1c58de None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:42.002122 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243ea16-55b0-4213-b171-17a45242b088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:42.002818 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243ea16-55b0-4213-b171-17a45242b088 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:42.003128 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243ea16-55b0-4213-b171-17a45242b088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:42.003431 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243ea16-55b0-4213-b171-17a45242b088 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:42.004174 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243ea16-55b0-4213-b171-17a45242b088 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:42.006102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243ea16-55b0-4213-b171-17a45242b088 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:42.088779 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:51:42.094592 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1224/2452] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:51:41 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:51:42.099404 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea44985-c524-4963-b894-5533d392a3fd admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:42.109714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea44985-c524-4963-b894-5533d392a3fd admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:42.114410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3991b30-eabd-447c-a02b-a807e8158193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:42.115320 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3991b30-eabd-447c-a02b-a807e8158193 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:42.115752 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3991b30-eabd-447c-a02b-a807e8158193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:42.115925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3991b30-eabd-447c-a02b-a807e8158193 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:42.116647 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3991b30-eabd-447c-a02b-a807e8158193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:42.118244 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3991b30-eabd-447c-a02b-a807e8158193 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:42.128775 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1229/2453] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:51:42.132872 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243ea16-55b0-4213-b171-17a45242b088 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:42.142998 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243ea16-55b0-4213-b171-17a45242b088 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:42.148375 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d7824-45a6-4f66-b9b4-6a178d6defc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:42.152633 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d7824-45a6-4f66-b9b4-6a178d6defc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:42.152633 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d7824-45a6-4f66-b9b4-6a178d6defc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:42.152633 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d7824-45a6-4f66-b9b4-6a178d6defc2 admin admin] PATH_INFO: `/v3/projects/87ceb058349c4f409d9e3793c93f85cb/users/c39953b927a54e7186d871857e27f11c/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:42.165271 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5d7824-45a6-4f66-b9b4-6a178d6defc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87ceb058349c4f409d9e3793c93f85cb, user_id=c39953b927a54e7186d871857e27f11c, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:42.168133 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5d7824-45a6-4f66-b9b4-6a178d6defc2 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:42.237921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1225/2454] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:51:42 2022] PUT /identity/v3/projects/87ceb058349c4f409d9e3793c93f85cb/users/c39953b927a54e7186d871857e27f11c/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:51:42.242314 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3991b30-eabd-447c-a02b-a807e8158193 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:42.252428 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3991b30-eabd-447c-a02b-a807e8158193 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:42.311057 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d0b34-55e1-42f3-b461-2fc4bd7c5a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:42.311710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d0b34-55e1-42f3-b461-2fc4bd7c5a80 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:42.311990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d0b34-55e1-42f3-b461-2fc4bd7c5a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:42.312263 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d0b34-55e1-42f3-b461-2fc4bd7c5a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:42.312982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d0b34-55e1-42f3-b461-2fc4bd7c5a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:42.314626 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d0b34-55e1-42f3-b461-2fc4bd7c5a80 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:42.325603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1230/2455] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:51:42.329835 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d7824-45a6-4f66-b9b4-6a178d6defc2 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:42.340969 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d7824-45a6-4f66-b9b4-6a178d6defc2 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:42.345695 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945242e-4a08-4cdb-ba50-465694938adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:42.346628 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945242e-4a08-4cdb-ba50-465694938adf admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:42.346915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945242e-4a08-4cdb-ba50-465694938adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:42.347221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945242e-4a08-4cdb-ba50-465694938adf admin admin] PATH_INFO: `/v3/projects/87ceb058349c4f409d9e3793c93f85cb/users/c39953b927a54e7186d871857e27f11c/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:42.349397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945242e-4a08-4cdb-ba50-465694938adf admin admin] RBAC: Authorizing `identity:create_grant(project_id=87ceb058349c4f409d9e3793c93f85cb, user_id=c39953b927a54e7186d871857e27f11c, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:42.352202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945242e-4a08-4cdb-ba50-465694938adf admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:42.362620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1226/2456] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:51:42 2022] PUT /identity/v3/projects/87ceb058349c4f409d9e3793c93f85cb/users/c39953b927a54e7186d871857e27f11c/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:51:42.372645 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2c666-bd6e-428b-9e16-7ca550c950c1 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:42.372930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2c666-bd6e-428b-9e16-7ca550c950c1 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:42.373208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2c666-bd6e-428b-9e16-7ca550c950c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:42.399914 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-5dc2c666-bd6e-428b-9e16-7ca550c950c1 None None] MFA Rules not processed for user `c39953b927a54e7186d871857e27f11c`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:51:42.401111 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-5dc2c666-bd6e-428b-9e16-7ca550c950c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:42.464043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:51:42.467597 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1231/2457] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:51:42 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:51:42.478837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945242e-4a08-4cdb-ba50-465694938adf admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:42.480573 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-5945242e-4a08-4cdb-ba50-465694938adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:42.537921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945242e-4a08-4cdb-ba50-465694938adf admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:42.541577 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f5eb2-de88-456e-b48a-c93e0e6e8d8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:42.542242 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f5eb2-de88-456e-b48a-c93e0e6e8d8f service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:42.542556 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f5eb2-de88-456e-b48a-c93e0e6e8d8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:42.542835 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f5eb2-de88-456e-b48a-c93e0e6e8d8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:42.551911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f5eb2-de88-456e-b48a-c93e0e6e8d8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:42.553332 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f5eb2-de88-456e-b48a-c93e0e6e8d8f service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:42.562708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1227/2458] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:51:42 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:51:42.735487 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2c666-bd6e-428b-9e16-7ca550c950c1 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:42.746683 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2c666-bd6e-428b-9e16-7ca550c950c1 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:42.750362 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b70980-364b-4390-b48e-8fe96dcd98aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:42.751106 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b70980-364b-4390-b48e-8fe96dcd98aa service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:42.751389 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b70980-364b-4390-b48e-8fe96dcd98aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:42.751667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b70980-364b-4390-b48e-8fe96dcd98aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:42.809239 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b70980-364b-4390-b48e-8fe96dcd98aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:42.810693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b70980-364b-4390-b48e-8fe96dcd98aa service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:42.820534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1232/2459] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:51:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:51:52.558484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=42,keystone:INSERT=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:51:52.815389 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=28 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:51:53.876214 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f5eb2-de88-456e-b48a-c93e0e6e8d8f service cinder] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:53.938457 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f5eb2-de88-456e-b48a-c93e0e6e8d8f service cinder] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:53.942935 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d387c1-9152-4687-9075-cce2614559cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:53.943823 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d387c1-9152-4687-9075-cce2614559cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:53.944104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d387c1-9152-4687-9075-cce2614559cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:53.944378 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d387c1-9152-4687-9075-cce2614559cf admin admin] PATH_INFO: `/v3/users/bc24ddc08f9f4466b43c35430b48b467` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:53.945611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d387c1-9152-4687-9075-cce2614559cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc24ddc08f9f4466b43c35430b48b467)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:53.947590 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d387c1-9152-4687-9075-cce2614559cf admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:54.109186 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-68d387c1-9152-4687-9075-cce2614559cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc24ddc08f9f4466b43c35430b48b467'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:54.116138 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-68d387c1-9152-4687-9075-cce2614559cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc24ddc08f9f4466b43c35430b48b467'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:54.116999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-68d387c1-9152-4687-9075-cce2614559cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc24ddc08f9f4466b43c35430b48b467'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:54.128679 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-68d387c1-9152-4687-9075-cce2614559cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc24ddc08f9f4466b43c35430b48b467'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:54.191751 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-68d387c1-9152-4687-9075-cce2614559cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc24ddc08f9f4466b43c35430b48b467'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:54.198989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1228/2460] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:51:53 2022] DELETE /identity/v3/users/bc24ddc08f9f4466b43c35430b48b467 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:51:54.203546 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feffcd7-d214-431e-aebf-e62da7be0d45 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:54.204047 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feffcd7-d214-431e-aebf-e62da7be0d45 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:54.204155 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feffcd7-d214-431e-aebf-e62da7be0d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:54.221849 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-3feffcd7-d214-431e-aebf-e62da7be0d45 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:51:54.223066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-3feffcd7-d214-431e-aebf-e62da7be0d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:54.289781 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1233/2461] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:51:54.299611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d387c1-9152-4687-9075-cce2614559cf admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:54.306167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-68d387c1-9152-4687-9075-cce2614559cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:54.359748 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d387c1-9152-4687-9075-cce2614559cf admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:54.363428 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a1e12f-907d-4478-980c-915a3e1f6c13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:54.364056 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1e12f-907d-4478-980c-915a3e1f6c13 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:54.364336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1e12f-907d-4478-980c-915a3e1f6c13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:54.364609 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1e12f-907d-4478-980c-915a3e1f6c13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:54.373965 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a1e12f-907d-4478-980c-915a3e1f6c13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:54.375409 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a1e12f-907d-4478-980c-915a3e1f6c13 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:54.385496 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1229/2462] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:51:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:51:54.799801 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feffcd7-d214-431e-aebf-e62da7be0d45 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:54.801238 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-3feffcd7-d214-431e-aebf-e62da7be0d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:54.815598 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feffcd7-d214-431e-aebf-e62da7be0d45 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:54.823435 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b80b5-e47f-499d-bede-4e622c52c0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:54.824339 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b80b5-e47f-499d-bede-4e622c52c0da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:54.824628 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b80b5-e47f-499d-bede-4e622c52c0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:54.824926 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b80b5-e47f-499d-bede-4e622c52c0da admin admin] PATH_INFO: `/v3/projects/1e8490a3f9ad48678c32d42cf10f1763` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:54.826185 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b80b5-e47f-499d-bede-4e622c52c0da admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e8490a3f9ad48678c32d42cf10f1763)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:54.828340 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b80b5-e47f-499d-bede-4e622c52c0da admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:54.999705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-0c4b80b5-e47f-499d-bede-4e622c52c0da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e8490a3f9ad48678c32d42cf10f1763', 'request_id': 'req-0c4b80b5-e47f-499d-bede-4e622c52c0da'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:55.055727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-0c4b80b5-e47f-499d-bede-4e622c52c0da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e8490a3f9ad48678c32d42cf10f1763', 'request_id': 'req-0c4b80b5-e47f-499d-bede-4e622c52c0da'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:55.063524 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-0c4b80b5-e47f-499d-bede-4e622c52c0da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e8490a3f9ad48678c32d42cf10f1763', 'request_id': 'req-0c4b80b5-e47f-499d-bede-4e622c52c0da'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:55.073880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-0c4b80b5-e47f-499d-bede-4e622c52c0da admin admin] The token cache is being invalidate because project 1e8490a3f9ad48678c32d42cf10f1763 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:51:55.074186 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-0c4b80b5-e47f-499d-bede-4e622c52c0da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:55.074852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-0c4b80b5-e47f-499d-bede-4e622c52c0da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:55.076419 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1234/2463] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:51:54 2022] DELETE /identity/v3/projects/1e8490a3f9ad48678c32d42cf10f1763 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:51:55.080451 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a1e12f-907d-4478-980c-915a3e1f6c13 service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:55.090124 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a1e12f-907d-4478-980c-915a3e1f6c13 service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:55.108424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:51:55.147327 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d2be4-26f0-4a07-892d-3329f1866e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:55.148385 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d2be4-26f0-4a07-892d-3329f1866e35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:55.148619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d2be4-26f0-4a07-892d-3329f1866e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:55.148932 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d2be4-26f0-4a07-892d-3329f1866e35 admin admin] PATH_INFO: `/v3/users/c39953b927a54e7186d871857e27f11c` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:55.150262 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d2be4-26f0-4a07-892d-3329f1866e35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c39953b927a54e7186d871857e27f11c)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:55.152917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d2be4-26f0-4a07-892d-3329f1866e35 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:55.292749 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-a61d2be4-26f0-4a07-892d-3329f1866e35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c39953b927a54e7186d871857e27f11c'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:55.300190 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-a61d2be4-26f0-4a07-892d-3329f1866e35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c39953b927a54e7186d871857e27f11c'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:55.300890 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-a61d2be4-26f0-4a07-892d-3329f1866e35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c39953b927a54e7186d871857e27f11c'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:55.314224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-a61d2be4-26f0-4a07-892d-3329f1866e35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c39953b927a54e7186d871857e27f11c'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:55.369050 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-a61d2be4-26f0-4a07-892d-3329f1866e35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c39953b927a54e7186d871857e27f11c'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:55.375807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1230/2464] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:51:55 2022] DELETE /identity/v3/users/c39953b927a54e7186d871857e27f11c => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:51:55.760309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b80b5-e47f-499d-bede-4e622c52c0da admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:55.761783 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-0c4b80b5-e47f-499d-bede-4e622c52c0da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:55.775452 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b80b5-e47f-499d-bede-4e622c52c0da admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:55.833828 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8b899-e672-4e20-af1b-7068efc88c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:55.839882 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8b899-e672-4e20-af1b-7068efc88c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:55.839882 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8b899-e672-4e20-af1b-7068efc88c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:55.839882 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8b899-e672-4e20-af1b-7068efc88c81 admin admin] PATH_INFO: `/v3/projects/87ceb058349c4f409d9e3793c93f85cb` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:55.839882 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f8b899-e672-4e20-af1b-7068efc88c81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87ceb058349c4f409d9e3793c93f85cb)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:55.839882 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f8b899-e672-4e20-af1b-7068efc88c81 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:55.977513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:51:55.997352 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-70f8b899-e672-4e20-af1b-7068efc88c81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87ceb058349c4f409d9e3793c93f85cb', 'request_id': 'req-70f8b899-e672-4e20-af1b-7068efc88c81'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:56.060935 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-70f8b899-e672-4e20-af1b-7068efc88c81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87ceb058349c4f409d9e3793c93f85cb', 'request_id': 'req-70f8b899-e672-4e20-af1b-7068efc88c81'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:56.067697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-70f8b899-e672-4e20-af1b-7068efc88c81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87ceb058349c4f409d9e3793c93f85cb', 'request_id': 'req-70f8b899-e672-4e20-af1b-7068efc88c81'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:56.079198 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-70f8b899-e672-4e20-af1b-7068efc88c81 admin admin] The token cache is being invalidate because project 87ceb058349c4f409d9e3793c93f85cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:51:56.079503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-70f8b899-e672-4e20-af1b-7068efc88c81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:56.080144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-70f8b899-e672-4e20-af1b-7068efc88c81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:56.081705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1235/2465] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:51:55 2022] DELETE /identity/v3/projects/87ceb058349c4f409d9e3793c93f85cb => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:51:56.087177 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b023305-ad0c-41ec-affe-480b96fd8f9c None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:56.087676 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b023305-ad0c-41ec-affe-480b96fd8f9c None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:56.087985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b023305-ad0c-41ec-affe-480b96fd8f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:56.107487 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-5b023305-ad0c-41ec-affe-480b96fd8f9c None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:51:56.108820 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-5b023305-ad0c-41ec-affe-480b96fd8f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:56.198418 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1231/2466] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:51:56.206784 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883a079-d82f-4bf5-b03d-cd6f413cd0ed None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:56.207111 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883a079-d82f-4bf5-b03d-cd6f413cd0ed None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:56.207450 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883a079-d82f-4bf5-b03d-cd6f413cd0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:56.233207 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-4883a079-d82f-4bf5-b03d-cd6f413cd0ed None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:51:56.234479 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-4883a079-d82f-4bf5-b03d-cd6f413cd0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:56.245354 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1236/2467] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:51:56.249672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b023305-ad0c-41ec-affe-480b96fd8f9c None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:56.261261 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b023305-ad0c-41ec-affe-480b96fd8f9c None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:56.266239 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28580478-5726-40e9-8f8a-8e571c1e1139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:56.267105 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28580478-5726-40e9-8f8a-8e571c1e1139 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:56.267462 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28580478-5726-40e9-8f8a-8e571c1e1139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:56.268645 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28580478-5726-40e9-8f8a-8e571c1e1139 admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:56.268645 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28580478-5726-40e9-8f8a-8e571c1e1139 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:51:56.269105 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28580478-5726-40e9-8f8a-8e571c1e1139 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:56.270996 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28580478-5726-40e9-8f8a-8e571c1e1139 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:56.290328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1232/2468] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 18:51:56.291570 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883a079-d82f-4bf5-b03d-cd6f413cd0ed None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:56.302008 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883a079-d82f-4bf5-b03d-cd6f413cd0ed None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:56.309799 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe5709-afd3-45c9-8ec1-fe88c1ffdb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:56.309799 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe5709-afd3-45c9-8ec1-fe88c1ffdb2d admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:56.309799 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe5709-afd3-45c9-8ec1-fe88c1ffdb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:56.310360 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe5709-afd3-45c9-8ec1-fe88c1ffdb2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:56.311258 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe5709-afd3-45c9-8ec1-fe88c1ffdb2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:56.313200 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe5709-afd3-45c9-8ec1-fe88c1ffdb2d admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:56.395353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1237/2469] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:51:56 2022] POST /identity/v3/projects => generated 369 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:51:56.399785 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28580478-5726-40e9-8f8a-8e571c1e1139 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:56.410335 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28580478-5726-40e9-8f8a-8e571c1e1139 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:56.438496 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:51:56.475953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df4fd3-9d80-4483-a595-e7dd7b490b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:56.477276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df4fd3-9d80-4483-a595-e7dd7b490b79 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:56.477776 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df4fd3-9d80-4483-a595-e7dd7b490b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:56.478047 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df4fd3-9d80-4483-a595-e7dd7b490b79 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:56.478780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df4fd3-9d80-4483-a595-e7dd7b490b79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:56.480721 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df4fd3-9d80-4483-a595-e7dd7b490b79 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:56.599565 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1233/2470] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:51:56 2022] POST /identity/v3/users => generated 285 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:51:56.605890 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe5709-afd3-45c9-8ec1-fe88c1ffdb2d admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:56.618669 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe5709-afd3-45c9-8ec1-fe88c1ffdb2d admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:56.623503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045271d7-95ef-4203-b23f-39fc3e436679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:56.624169 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045271d7-95ef-4203-b23f-39fc3e436679 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:56.624453 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045271d7-95ef-4203-b23f-39fc3e436679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:56.624725 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045271d7-95ef-4203-b23f-39fc3e436679 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:56.625563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045271d7-95ef-4203-b23f-39fc3e436679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:56.627230 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045271d7-95ef-4203-b23f-39fc3e436679 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:56.637944 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1238/2471] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:51:56.642209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df4fd3-9d80-4483-a595-e7dd7b490b79 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:56.653590 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df4fd3-9d80-4483-a595-e7dd7b490b79 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:56.658502 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2973159-1917-418e-9fee-e341289e890f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:56.659160 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2973159-1917-418e-9fee-e341289e890f admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:56.662530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2973159-1917-418e-9fee-e341289e890f admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:56.662530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2973159-1917-418e-9fee-e341289e890f admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:56.662530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2973159-1917-418e-9fee-e341289e890f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:56.662530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2973159-1917-418e-9fee-e341289e890f admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:56.674446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1234/2472] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:51:56.678635 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045271d7-95ef-4203-b23f-39fc3e436679 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:56.689399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045271d7-95ef-4203-b23f-39fc3e436679 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:56.694123 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d47b2-cf09-496b-9cfb-8b78d4a39913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:56.695040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d47b2-cf09-496b-9cfb-8b78d4a39913 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:56.695325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d47b2-cf09-496b-9cfb-8b78d4a39913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:56.695631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d47b2-cf09-496b-9cfb-8b78d4a39913 admin admin] PATH_INFO: `/v3/projects/1cc286b8b4514726a8b548f77acd1d20/users/3f6dcc49c80840c786a038b5002c7a8c/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:56.711477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d47b2-cf09-496b-9cfb-8b78d4a39913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cc286b8b4514726a8b548f77acd1d20, user_id=3f6dcc49c80840c786a038b5002c7a8c, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:56.714361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d47b2-cf09-496b-9cfb-8b78d4a39913 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:56.783073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1239/2473] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:51:56 2022] PUT /identity/v3/projects/1cc286b8b4514726a8b548f77acd1d20/users/3f6dcc49c80840c786a038b5002c7a8c/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:51:56.795127 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92480f59-31cf-49f6-9191-2acf7f0fa066 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:56.795419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92480f59-31cf-49f6-9191-2acf7f0fa066 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:56.795700 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92480f59-31cf-49f6-9191-2acf7f0fa066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:56.838113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-92480f59-31cf-49f6-9191-2acf7f0fa066 None None] MFA Rules not processed for user `3f6dcc49c80840c786a038b5002c7a8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:51:56.839382 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-92480f59-31cf-49f6-9191-2acf7f0fa066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:56.918556 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1235/2474] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:51:56 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:51:56.923301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d47b2-cf09-496b-9cfb-8b78d4a39913 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:57.001878 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d47b2-cf09-496b-9cfb-8b78d4a39913 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:57.011462 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e7af63-fb83-46da-88c1-c759fb2b34cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:57.012364 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7af63-fb83-46da-88c1-c759fb2b34cf admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:57.012754 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7af63-fb83-46da-88c1-c759fb2b34cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:57.013127 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7af63-fb83-46da-88c1-c759fb2b34cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:57.015079 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e7af63-fb83-46da-88c1-c759fb2b34cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:57.018160 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e7af63-fb83-46da-88c1-c759fb2b34cf admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:57.110130 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1240/2475] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:51:56 2022] POST /identity/v3/projects => generated 371 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:51:57.113748 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92480f59-31cf-49f6-9191-2acf7f0fa066 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:57.125362 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92480f59-31cf-49f6-9191-2acf7f0fa066 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:57.191872 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e13ecb-3c42-4548-8651-bc12cba010fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:57.192655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e13ecb-3c42-4548-8651-bc12cba010fe admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:57.193017 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e13ecb-3c42-4548-8651-bc12cba010fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:57.193371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e13ecb-3c42-4548-8651-bc12cba010fe admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:57.194194 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e13ecb-3c42-4548-8651-bc12cba010fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:57.197700 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e13ecb-3c42-4548-8651-bc12cba010fe admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:57.277403 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1236/2476] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:51:57 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:51:57.278618 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e7af63-fb83-46da-88c1-c759fb2b34cf admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:57.289880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e7af63-fb83-46da-88c1-c759fb2b34cf admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:57.294989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807923c-1b38-4740-8387-a30eadef8011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:57.295704 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807923c-1b38-4740-8387-a30eadef8011 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:57.295988 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807923c-1b38-4740-8387-a30eadef8011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:57.296262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807923c-1b38-4740-8387-a30eadef8011 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:57.296990 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807923c-1b38-4740-8387-a30eadef8011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:57.298657 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807923c-1b38-4740-8387-a30eadef8011 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:57.309342 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1241/2477] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:51:57.313819 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e13ecb-3c42-4548-8651-bc12cba010fe admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:57.325295 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e13ecb-3c42-4548-8651-bc12cba010fe admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:57.330439 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320073c-ad77-4d7a-afd4-cb3013d69218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:57.331710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320073c-ad77-4d7a-afd4-cb3013d69218 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:57.332116 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320073c-ad77-4d7a-afd4-cb3013d69218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:57.336266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320073c-ad77-4d7a-afd4-cb3013d69218 admin admin] PATH_INFO: `/v3/projects/36013df065b44ac9a3feb50f129441a1/users/49384042b0bd4362b51fe74bd434f580/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:57.349438 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a320073c-ad77-4d7a-afd4-cb3013d69218 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36013df065b44ac9a3feb50f129441a1, user_id=49384042b0bd4362b51fe74bd434f580, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:57.352170 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a320073c-ad77-4d7a-afd4-cb3013d69218 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:57.426978 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1237/2478] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:51:57 2022] PUT /identity/v3/projects/36013df065b44ac9a3feb50f129441a1/users/49384042b0bd4362b51fe74bd434f580/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:51:57.431065 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807923c-1b38-4740-8387-a30eadef8011 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:57.441002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807923c-1b38-4740-8387-a30eadef8011 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:57.463907 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:51:57.498547 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5418f2-71b5-4101-9825-f24c9addd059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:57.499199 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5418f2-71b5-4101-9825-f24c9addd059 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:57.499501 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5418f2-71b5-4101-9825-f24c9addd059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:57.499771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5418f2-71b5-4101-9825-f24c9addd059 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:57.500483 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5418f2-71b5-4101-9825-f24c9addd059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:57.502066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5418f2-71b5-4101-9825-f24c9addd059 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:57.512064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1242/2479] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:51:57.516404 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320073c-ad77-4d7a-afd4-cb3013d69218 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:57.521918 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:51:57.529331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320073c-ad77-4d7a-afd4-cb3013d69218 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:57.533986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652165d-eee7-4f4f-b036-219129997500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:57.534908 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652165d-eee7-4f4f-b036-219129997500 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:57.536061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652165d-eee7-4f4f-b036-219129997500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:57.536061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652165d-eee7-4f4f-b036-219129997500 admin admin] PATH_INFO: `/v3/projects/36013df065b44ac9a3feb50f129441a1/users/49384042b0bd4362b51fe74bd434f580/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:57.537578 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652165d-eee7-4f4f-b036-219129997500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36013df065b44ac9a3feb50f129441a1, user_id=49384042b0bd4362b51fe74bd434f580, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:57.540411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652165d-eee7-4f4f-b036-219129997500 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:57.551111 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1238/2480] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:51:57 2022] PUT /identity/v3/projects/36013df065b44ac9a3feb50f129441a1/users/49384042b0bd4362b51fe74bd434f580/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:51:57.561892 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f936c66-92b2-4d2f-bc73-b9a3e5ad973d None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:57.562267 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f936c66-92b2-4d2f-bc73-b9a3e5ad973d None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:57.562530 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f936c66-92b2-4d2f-bc73-b9a3e5ad973d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:57.588865 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-4f936c66-92b2-4d2f-bc73-b9a3e5ad973d None None] MFA Rules not processed for user `49384042b0bd4362b51fe74bd434f580`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:51:57.590112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-4f936c66-92b2-4d2f-bc73-b9a3e5ad973d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:57.657620 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1243/2481] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:51:57 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:51:57.670474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652165d-eee7-4f4f-b036-219129997500 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:57.672175 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-0652165d-eee7-4f4f-b036-219129997500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:57.730423 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652165d-eee7-4f4f-b036-219129997500 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:57.734179 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc74a50d-0575-4551-a899-803301983d45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:57.734867 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74a50d-0575-4551-a899-803301983d45 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:57.735145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74a50d-0575-4551-a899-803301983d45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:57.735416 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74a50d-0575-4551-a899-803301983d45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:57.748343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74a50d-0575-4551-a899-803301983d45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:57.749789 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74a50d-0575-4551-a899-803301983d45 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:57.760507 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1239/2482] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:51:57 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:51:58.655719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f936c66-92b2-4d2f-bc73-b9a3e5ad973d None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:58.723039 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f936c66-92b2-4d2f-bc73-b9a3e5ad973d None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:58.728496 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a20afb-c3dc-4e2f-94c8-ffd676ee7098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:58.729361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a20afb-c3dc-4e2f-94c8-ffd676ee7098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:58.729638 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a20afb-c3dc-4e2f-94c8-ffd676ee7098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:58.729912 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a20afb-c3dc-4e2f-94c8-ffd676ee7098 admin admin] PATH_INFO: `/v3/users/3f6dcc49c80840c786a038b5002c7a8c` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:58.731108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a20afb-c3dc-4e2f-94c8-ffd676ee7098 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f6dcc49c80840c786a038b5002c7a8c)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:58.733051 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a20afb-c3dc-4e2f-94c8-ffd676ee7098 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:58.942621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-79a20afb-c3dc-4e2f-94c8-ffd676ee7098 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f6dcc49c80840c786a038b5002c7a8c'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:58.943981 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:51:58.952287 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-79a20afb-c3dc-4e2f-94c8-ffd676ee7098 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f6dcc49c80840c786a038b5002c7a8c'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:58.953104 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-79a20afb-c3dc-4e2f-94c8-ffd676ee7098 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f6dcc49c80840c786a038b5002c7a8c'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:58.953620 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-79a20afb-c3dc-4e2f-94c8-ffd676ee7098 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f6dcc49c80840c786a038b5002c7a8c'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:58.965873 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-79a20afb-c3dc-4e2f-94c8-ffd676ee7098 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f6dcc49c80840c786a038b5002c7a8c'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:51:59.066033 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1244/2483] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:51:58 2022] DELETE /identity/v3/users/3f6dcc49c80840c786a038b5002c7a8c => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:51:59.070507 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a703f38-8d97-44cf-9060-f4d3854ec9da None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:59.071006 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a703f38-8d97-44cf-9060-f4d3854ec9da None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:59.071103 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a703f38-8d97-44cf-9060-f4d3854ec9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:59.088081 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-2a703f38-8d97-44cf-9060-f4d3854ec9da None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:51:59.089270 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-2a703f38-8d97-44cf-9060-f4d3854ec9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:59.146788 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1240/2484] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:51:59.156907 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a20afb-c3dc-4e2f-94c8-ffd676ee7098 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:59.158328 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-79a20afb-c3dc-4e2f-94c8-ffd676ee7098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:59.219443 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a20afb-c3dc-4e2f-94c8-ffd676ee7098 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:59.223585 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888a73a-ef99-4b3c-9df4-f0012dde1f46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:59.224374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888a73a-ef99-4b3c-9df4-f0012dde1f46 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:59.225099 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888a73a-ef99-4b3c-9df4-f0012dde1f46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:59.225674 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888a73a-ef99-4b3c-9df4-f0012dde1f46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:59.239848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b888a73a-ef99-4b3c-9df4-f0012dde1f46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:59.243185 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b888a73a-ef99-4b3c-9df4-f0012dde1f46 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:59.253169 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1245/2485] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:51:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:51:59.805948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a703f38-8d97-44cf-9060-f4d3854ec9da None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:51:59.807563 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-2a703f38-8d97-44cf-9060-f4d3854ec9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:51:59.820896 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a703f38-8d97-44cf-9060-f4d3854ec9da None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:51:59.825991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81446051-d57a-4d4d-8dfb-d6f3a4c02492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:51:59.831990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81446051-d57a-4d4d-8dfb-d6f3a4c02492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:51:59.831990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81446051-d57a-4d4d-8dfb-d6f3a4c02492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:51:59.831990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81446051-d57a-4d4d-8dfb-d6f3a4c02492 admin admin] PATH_INFO: `/v3/projects/1cc286b8b4514726a8b548f77acd1d20` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:51:59.831990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81446051-d57a-4d4d-8dfb-d6f3a4c02492 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cc286b8b4514726a8b548f77acd1d20)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:51:59.831990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81446051-d57a-4d4d-8dfb-d6f3a4c02492 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:51:59.980896 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-81446051-d57a-4d4d-8dfb-d6f3a4c02492 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cc286b8b4514726a8b548f77acd1d20', 'request_id': 'req-81446051-d57a-4d4d-8dfb-d6f3a4c02492'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:00.065850 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-81446051-d57a-4d4d-8dfb-d6f3a4c02492 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cc286b8b4514726a8b548f77acd1d20', 'request_id': 'req-81446051-d57a-4d4d-8dfb-d6f3a4c02492'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:00.073554 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-81446051-d57a-4d4d-8dfb-d6f3a4c02492 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cc286b8b4514726a8b548f77acd1d20', 'request_id': 'req-81446051-d57a-4d4d-8dfb-d6f3a4c02492'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:00.092356 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-81446051-d57a-4d4d-8dfb-d6f3a4c02492 admin admin] The token cache is being invalidate because project 1cc286b8b4514726a8b548f77acd1d20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:52:00.092356 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-81446051-d57a-4d4d-8dfb-d6f3a4c02492 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:00.092356 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-81446051-d57a-4d4d-8dfb-d6f3a4c02492 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:00.092356 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1241/2486] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:51:59 2022] DELETE /identity/v3/projects/1cc286b8b4514726a8b548f77acd1d20 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:00.092770 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888a73a-ef99-4b3c-9df4-f0012dde1f46 service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:00.100900 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888a73a-ef99-4b3c-9df4-f0012dde1f46 service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:00.156612 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978ed06-3489-4796-aebd-e12f609f9474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:00.157461 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978ed06-3489-4796-aebd-e12f609f9474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:00.157742 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978ed06-3489-4796-aebd-e12f609f9474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:00.158016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978ed06-3489-4796-aebd-e12f609f9474 admin admin] PATH_INFO: `/v3/users/49384042b0bd4362b51fe74bd434f580` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:00.159202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c978ed06-3489-4796-aebd-e12f609f9474 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49384042b0bd4362b51fe74bd434f580)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:00.161229 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c978ed06-3489-4796-aebd-e12f609f9474 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:00.360510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-c978ed06-3489-4796-aebd-e12f609f9474 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49384042b0bd4362b51fe74bd434f580'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:00.368421 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-c978ed06-3489-4796-aebd-e12f609f9474 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49384042b0bd4362b51fe74bd434f580'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:00.369085 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-c978ed06-3489-4796-aebd-e12f609f9474 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49384042b0bd4362b51fe74bd434f580'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:00.369688 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-c978ed06-3489-4796-aebd-e12f609f9474 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49384042b0bd4362b51fe74bd434f580'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:00.379949 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:00.382226 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-c978ed06-3489-4796-aebd-e12f609f9474 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49384042b0bd4362b51fe74bd434f580'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:00.460323 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1246/2487] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:52:00 2022] DELETE /identity/v3/users/49384042b0bd4362b51fe74bd434f580 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:00.858363 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81446051-d57a-4d4d-8dfb-d6f3a4c02492 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:00.859948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-81446051-d57a-4d4d-8dfb-d6f3a4c02492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:00.873931 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81446051-d57a-4d4d-8dfb-d6f3a4c02492 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:00.886615 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:00.938654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b66e2-165b-4932-9681-bdd498bca168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:00.939522 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b66e2-165b-4932-9681-bdd498bca168 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:00.939807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b66e2-165b-4932-9681-bdd498bca168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:00.940085 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b66e2-165b-4932-9681-bdd498bca168 admin admin] PATH_INFO: `/v3/projects/36013df065b44ac9a3feb50f129441a1` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:00.941268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b66e2-165b-4932-9681-bdd498bca168 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36013df065b44ac9a3feb50f129441a1)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:00.943208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b66e2-165b-4932-9681-bdd498bca168 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:01.043650 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-6a0b66e2-165b-4932-9681-bdd498bca168 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36013df065b44ac9a3feb50f129441a1', 'request_id': 'req-6a0b66e2-165b-4932-9681-bdd498bca168'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:01.152651 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-6a0b66e2-165b-4932-9681-bdd498bca168 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36013df065b44ac9a3feb50f129441a1', 'request_id': 'req-6a0b66e2-165b-4932-9681-bdd498bca168'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:01.159315 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-6a0b66e2-165b-4932-9681-bdd498bca168 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36013df065b44ac9a3feb50f129441a1', 'request_id': 'req-6a0b66e2-165b-4932-9681-bdd498bca168'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:01.169967 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-6a0b66e2-165b-4932-9681-bdd498bca168 admin admin] The token cache is being invalidate because project 36013df065b44ac9a3feb50f129441a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:52:01.170252 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-6a0b66e2-165b-4932-9681-bdd498bca168 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:01.170988 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-6a0b66e2-165b-4932-9681-bdd498bca168 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:01.172623 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1242/2488] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:52:00 2022] DELETE /identity/v3/projects/36013df065b44ac9a3feb50f129441a1 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:01.178526 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193408ef-6174-4b69-adf7-ed32b8bb03b5 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:01.183493 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193408ef-6174-4b69-adf7-ed32b8bb03b5 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:01.183493 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193408ef-6174-4b69-adf7-ed32b8bb03b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:01.198346 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-193408ef-6174-4b69-adf7-ed32b8bb03b5 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:01.199579 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-193408ef-6174-4b69-adf7-ed32b8bb03b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:01.259414 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1247/2489] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:01.268497 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b766a1e-87be-40af-9841-691e1831a51a None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:01.268813 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b766a1e-87be-40af-9841-691e1831a51a None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:01.269122 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b766a1e-87be-40af-9841-691e1831a51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:01.288073 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-8b766a1e-87be-40af-9841-691e1831a51a None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:01.289300 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-8b766a1e-87be-40af-9841-691e1831a51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:01.297214 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1243/2490] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:01.301495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193408ef-6174-4b69-adf7-ed32b8bb03b5 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:01.312010 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193408ef-6174-4b69-adf7-ed32b8bb03b5 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:01.316744 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f031a3-171c-44b6-aafe-9ae4990f81db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:01.317384 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f031a3-171c-44b6-aafe-9ae4990f81db admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:01.317666 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f031a3-171c-44b6-aafe-9ae4990f81db admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:01.317940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f031a3-171c-44b6-aafe-9ae4990f81db admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:01.318707 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f031a3-171c-44b6-aafe-9ae4990f81db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:52:01.319011 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f031a3-171c-44b6-aafe-9ae4990f81db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:01.321164 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f031a3-171c-44b6-aafe-9ae4990f81db admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:01.332761 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1248/2491] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:52:01.337039 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b766a1e-87be-40af-9841-691e1831a51a None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:01.346818 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b766a1e-87be-40af-9841-691e1831a51a None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:01.351682 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f66ac3-c749-49d9-98aa-ce347a285746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:01.352268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f66ac3-c749-49d9-98aa-ce347a285746 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:01.352552 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f66ac3-c749-49d9-98aa-ce347a285746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:01.352884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f66ac3-c749-49d9-98aa-ce347a285746 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:01.353590 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f66ac3-c749-49d9-98aa-ce347a285746 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:01.355542 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f66ac3-c749-49d9-98aa-ce347a285746 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:01.467971 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1244/2492] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:52:01 2022] POST /identity/v3/projects => generated 409 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:01.472053 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f031a3-171c-44b6-aafe-9ae4990f81db admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:01.483759 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f031a3-171c-44b6-aafe-9ae4990f81db admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:01.540491 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cac82f-2473-4107-9013-9062a4251118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:01.541152 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cac82f-2473-4107-9013-9062a4251118 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:01.541433 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cac82f-2473-4107-9013-9062a4251118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:01.541714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cac82f-2473-4107-9013-9062a4251118 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:01.542456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cac82f-2473-4107-9013-9062a4251118 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:01.544376 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cac82f-2473-4107-9013-9062a4251118 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:01.653624 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1249/2493] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:52:01 2022] POST /identity/v3/users => generated 305 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:01.656505 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f66ac3-c749-49d9-98aa-ce347a285746 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:01.670166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f66ac3-c749-49d9-98aa-ce347a285746 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:01.676786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6498e-034e-4c8e-834d-57d354392c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:01.676786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6498e-034e-4c8e-834d-57d354392c62 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:01.676786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6498e-034e-4c8e-834d-57d354392c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:01.676786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6498e-034e-4c8e-834d-57d354392c62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:01.677441 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6498e-034e-4c8e-834d-57d354392c62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:01.679143 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6498e-034e-4c8e-834d-57d354392c62 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:01.690846 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1245/2494] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:01.694639 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cac82f-2473-4107-9013-9062a4251118 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:01.704707 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cac82f-2473-4107-9013-9062a4251118 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:01.709296 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df2271c-7c9e-4e08-848d-45a2733120a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:01.714679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2271c-7c9e-4e08-848d-45a2733120a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:01.714679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2271c-7c9e-4e08-848d-45a2733120a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:01.714679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df2271c-7c9e-4e08-848d-45a2733120a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:01.714679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2271c-7c9e-4e08-848d-45a2733120a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:01.714679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df2271c-7c9e-4e08-848d-45a2733120a9 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:01.724002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1250/2495] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:01.729234 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6498e-034e-4c8e-834d-57d354392c62 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:01.739441 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6498e-034e-4c8e-834d-57d354392c62 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:01.745338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30012a3d-3d8c-4ba8-9e68-d06395dc07ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:01.746184 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30012a3d-3d8c-4ba8-9e68-d06395dc07ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:01.746622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30012a3d-3d8c-4ba8-9e68-d06395dc07ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:01.746925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30012a3d-3d8c-4ba8-9e68-d06395dc07ff admin admin] PATH_INFO: `/v3/projects/857fa274a1594a258bb662e743776f46/users/f7fd83a789aa4a5b87c1bdfdc3e20da3/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:01.762069 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30012a3d-3d8c-4ba8-9e68-d06395dc07ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=857fa274a1594a258bb662e743776f46, user_id=f7fd83a789aa4a5b87c1bdfdc3e20da3, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:01.764774 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30012a3d-3d8c-4ba8-9e68-d06395dc07ff admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:01.896790 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1246/2496] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:01 2022] PUT /identity/v3/projects/857fa274a1594a258bb662e743776f46/users/f7fd83a789aa4a5b87c1bdfdc3e20da3/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:01.907914 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7abece-5e2f-4572-843a-569df04ed97c None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:01.908540 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7abece-5e2f-4572-843a-569df04ed97c None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:01.908872 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7abece-5e2f-4572-843a-569df04ed97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:01.949137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-fc7abece-5e2f-4572-843a-569df04ed97c None None] MFA Rules not processed for user `f7fd83a789aa4a5b87c1bdfdc3e20da3`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:01.955139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-fc7abece-5e2f-4572-843a-569df04ed97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:01.995254 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:02.018354 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1251/2497] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:01 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:02.022805 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30012a3d-3d8c-4ba8-9e68-d06395dc07ff admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:02.069493 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:02.086978 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30012a3d-3d8c-4ba8-9e68-d06395dc07ff admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:02.091647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9096c-d9c6-4dc2-b992-d58d86044db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:02.092294 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9096c-d9c6-4dc2-b992-d58d86044db0 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:02.092578 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9096c-d9c6-4dc2-b992-d58d86044db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:02.092852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9096c-d9c6-4dc2-b992-d58d86044db0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:02.093538 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9096c-d9c6-4dc2-b992-d58d86044db0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:02.095470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9096c-d9c6-4dc2-b992-d58d86044db0 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:02.212161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1247/2498] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:52:02 2022] POST /identity/v3/projects => generated 407 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:02.216277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7abece-5e2f-4572-843a-569df04ed97c None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:02.227656 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7abece-5e2f-4572-843a-569df04ed97c None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:02.283377 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5dffa-d01d-4b5e-b2b8-dce73e697492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:02.283958 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5dffa-d01d-4b5e-b2b8-dce73e697492 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:02.284243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5dffa-d01d-4b5e-b2b8-dce73e697492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:02.284518 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5dffa-d01d-4b5e-b2b8-dce73e697492 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:02.285197 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5dffa-d01d-4b5e-b2b8-dce73e697492 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:02.287158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5dffa-d01d-4b5e-b2b8-dce73e697492 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:02.404840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1252/2499] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:52:02 2022] POST /identity/v3/users => generated 310 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:02.414646 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9096c-d9c6-4dc2-b992-d58d86044db0 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:02.424964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9096c-d9c6-4dc2-b992-d58d86044db0 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:02.429627 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393d0e3-cc8b-48a5-9234-fc753f0e2d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:02.430278 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393d0e3-cc8b-48a5-9234-fc753f0e2d1a admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:02.430615 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393d0e3-cc8b-48a5-9234-fc753f0e2d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:02.430918 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393d0e3-cc8b-48a5-9234-fc753f0e2d1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:02.431634 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393d0e3-cc8b-48a5-9234-fc753f0e2d1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:02.433445 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393d0e3-cc8b-48a5-9234-fc753f0e2d1a admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:02.444118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1248/2500] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:02.449024 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5dffa-d01d-4b5e-b2b8-dce73e697492 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:02.459166 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5dffa-d01d-4b5e-b2b8-dce73e697492 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:02.464400 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37db2a-b7fc-4c85-b03d-f3e1f70e612f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:02.465270 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37db2a-b7fc-4c85-b03d-f3e1f70e612f admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:02.465550 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37db2a-b7fc-4c85-b03d-f3e1f70e612f admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:02.465830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37db2a-b7fc-4c85-b03d-f3e1f70e612f admin admin] PATH_INFO: `/v3/projects/2f666ce0759440168580334451f7e377/users/8bb2911660254e3f89326ef680e0e4d0/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:02.480977 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37db2a-b7fc-4c85-b03d-f3e1f70e612f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f666ce0759440168580334451f7e377, user_id=8bb2911660254e3f89326ef680e0e4d0, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:02.483949 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37db2a-b7fc-4c85-b03d-f3e1f70e612f admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:02.592945 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1253/2501] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:02 2022] PUT /identity/v3/projects/2f666ce0759440168580334451f7e377/users/8bb2911660254e3f89326ef680e0e4d0/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:02.597194 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393d0e3-cc8b-48a5-9234-fc753f0e2d1a admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:02.607044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393d0e3-cc8b-48a5-9234-fc753f0e2d1a admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:02.662621 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79224a-007d-4ff1-8bf8-439969fc6a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:02.663275 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79224a-007d-4ff1-8bf8-439969fc6a6c admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:02.663557 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79224a-007d-4ff1-8bf8-439969fc6a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:02.664105 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79224a-007d-4ff1-8bf8-439969fc6a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:02.664580 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79224a-007d-4ff1-8bf8-439969fc6a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:02.666176 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79224a-007d-4ff1-8bf8-439969fc6a6c admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:02.676469 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1249/2502] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:02.681198 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37db2a-b7fc-4c85-b03d-f3e1f70e612f admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:02.691328 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37db2a-b7fc-4c85-b03d-f3e1f70e612f admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:02.696848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b574c9-610e-4576-8d56-3814000593f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:02.697711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b574c9-610e-4576-8d56-3814000593f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:02.697988 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b574c9-610e-4576-8d56-3814000593f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:02.698264 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b574c9-610e-4576-8d56-3814000593f9 admin admin] PATH_INFO: `/v3/projects/2f666ce0759440168580334451f7e377/users/8bb2911660254e3f89326ef680e0e4d0/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:02.700229 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b574c9-610e-4576-8d56-3814000593f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f666ce0759440168580334451f7e377, user_id=8bb2911660254e3f89326ef680e0e4d0, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:02.702894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b574c9-610e-4576-8d56-3814000593f9 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:02.713826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1254/2503] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:02 2022] PUT /identity/v3/projects/2f666ce0759440168580334451f7e377/users/8bb2911660254e3f89326ef680e0e4d0/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:02.723740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48cf95-8295-4fce-9d2d-a1e106c9ed6c None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:02.724062 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48cf95-8295-4fce-9d2d-a1e106c9ed6c None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:02.724339 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48cf95-8295-4fce-9d2d-a1e106c9ed6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:02.751949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-4c48cf95-8295-4fce-9d2d-a1e106c9ed6c None None] MFA Rules not processed for user `8bb2911660254e3f89326ef680e0e4d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:02.753151 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-4c48cf95-8295-4fce-9d2d-a1e106c9ed6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:02.821555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1250/2504] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:02 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:02.832178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b574c9-610e-4576-8d56-3814000593f9 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:02.833856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c8b574c9-610e-4576-8d56-3814000593f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:02.891733 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b574c9-610e-4576-8d56-3814000593f9 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:02.895963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84b450-2e42-4d6c-b0ad-ce15bc56068b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:02.896605 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84b450-2e42-4d6c-b0ad-ce15bc56068b service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:02.896887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84b450-2e42-4d6c-b0ad-ce15bc56068b service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:02.897171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84b450-2e42-4d6c-b0ad-ce15bc56068b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:02.906666 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84b450-2e42-4d6c-b0ad-ce15bc56068b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:02.908089 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d84b450-2e42-4d6c-b0ad-ce15bc56068b service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:02.917474 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1255/2505] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:02 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:52:03.022175 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48cf95-8295-4fce-9d2d-a1e106c9ed6c None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:03.023660 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-4c48cf95-8295-4fce-9d2d-a1e106c9ed6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:03.066801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:03.081734 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48cf95-8295-4fce-9d2d-a1e106c9ed6c None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:03.085426 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a2712-610f-46cd-b70d-e270fe995a07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:03.087188 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a2712-610f-46cd-b70d-e270fe995a07 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:03.091753 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a2712-610f-46cd-b70d-e270fe995a07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:03.091753 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a2712-610f-46cd-b70d-e270fe995a07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:03.091753 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-356a2712-610f-46cd-b70d-e270fe995a07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:03.156240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a2712-610f-46cd-b70d-e270fe995a07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:03.157591 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a2712-610f-46cd-b70d-e270fe995a07 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:03.168912 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1251/2506] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:52:03.275728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84b450-2e42-4d6c-b0ad-ce15bc56068b service cinder] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:03.284248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84b450-2e42-4d6c-b0ad-ce15bc56068b service cinder] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:03.288020 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0205ba2-0a9a-45ea-b72a-01e70aa619e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:03.288667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0205ba2-0a9a-45ea-b72a-01e70aa619e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:03.288955 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0205ba2-0a9a-45ea-b72a-01e70aa619e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:03.289250 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0205ba2-0a9a-45ea-b72a-01e70aa619e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:03.323246 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:03.346492 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0205ba2-0a9a-45ea-b72a-01e70aa619e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:03.347948 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0205ba2-0a9a-45ea-b72a-01e70aa619e0 service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:03.357810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1256/2507] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:03 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:52:07.124074 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a2712-610f-46cd-b70d-e270fe995a07 service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:07.125231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-356a2712-610f-46cd-b70d-e270fe995a07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:07.137814 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a2712-610f-46cd-b70d-e270fe995a07 service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:07.142419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c2146-c11b-4bd1-b656-fa2d75537ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:07.143290 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c2146-c11b-4bd1-b656-fa2d75537ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:07.143571 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c2146-c11b-4bd1-b656-fa2d75537ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:07.143842 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c2146-c11b-4bd1-b656-fa2d75537ad4 admin admin] PATH_INFO: `/v3/users/2fc6711e48be4d4ab33695e06c7227ab` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:07.145029 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c2146-c11b-4bd1-b656-fa2d75537ad4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fc6711e48be4d4ab33695e06c7227ab)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:07.146990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c2146-c11b-4bd1-b656-fa2d75537ad4 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:07.388530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-d24c2146-c11b-4bd1-b656-fa2d75537ad4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fc6711e48be4d4ab33695e06c7227ab'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:07.395405 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-d24c2146-c11b-4bd1-b656-fa2d75537ad4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fc6711e48be4d4ab33695e06c7227ab'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:07.396066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-d24c2146-c11b-4bd1-b656-fa2d75537ad4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fc6711e48be4d4ab33695e06c7227ab'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:07.408342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-d24c2146-c11b-4bd1-b656-fa2d75537ad4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fc6711e48be4d4ab33695e06c7227ab'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:07.500974 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-d24c2146-c11b-4bd1-b656-fa2d75537ad4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fc6711e48be4d4ab33695e06c7227ab'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:07.507300 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1252/2508] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:52:07 2022] DELETE /identity/v3/users/2fc6711e48be4d4ab33695e06c7227ab => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:07.767246 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0205ba2-0a9a-45ea-b72a-01e70aa619e0 service cinder] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:07.779525 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0205ba2-0a9a-45ea-b72a-01e70aa619e0 service cinder] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:07.845029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf8958-b817-40a3-8aab-d150bfff500d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:07.845882 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf8958-b817-40a3-8aab-d150bfff500d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:07.846336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf8958-b817-40a3-8aab-d150bfff500d admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:07.846644 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf8958-b817-40a3-8aab-d150bfff500d admin admin] PATH_INFO: `/v3/projects/0f19df10a9c948ce9c2e0dfa29532e5a` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:07.847835 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf8958-b817-40a3-8aab-d150bfff500d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f19df10a9c948ce9c2e0dfa29532e5a)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:07.850566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf8958-b817-40a3-8aab-d150bfff500d admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:07.981859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b0bf8958-b817-40a3-8aab-d150bfff500d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f19df10a9c948ce9c2e0dfa29532e5a', 'request_id': 'req-b0bf8958-b817-40a3-8aab-d150bfff500d'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:08.055888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b0bf8958-b817-40a3-8aab-d150bfff500d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f19df10a9c948ce9c2e0dfa29532e5a', 'request_id': 'req-b0bf8958-b817-40a3-8aab-d150bfff500d'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:08.062769 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b0bf8958-b817-40a3-8aab-d150bfff500d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f19df10a9c948ce9c2e0dfa29532e5a', 'request_id': 'req-b0bf8958-b817-40a3-8aab-d150bfff500d'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:08.073882 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b0bf8958-b817-40a3-8aab-d150bfff500d admin admin] The token cache is being invalidate because project 0f19df10a9c948ce9c2e0dfa29532e5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:52:08.074174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b0bf8958-b817-40a3-8aab-d150bfff500d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:08.074857 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b0bf8958-b817-40a3-8aab-d150bfff500d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:08.076409 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1257/2509] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:52:07 2022] DELETE /identity/v3/projects/0f19df10a9c948ce9c2e0dfa29532e5a => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:08.081691 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8016174-1fb7-4deb-ae7e-0e4692dbf0f1 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:08.082033 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8016174-1fb7-4deb-ae7e-0e4692dbf0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:08.082322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8016174-1fb7-4deb-ae7e-0e4692dbf0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:08.100447 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-d8016174-1fb7-4deb-ae7e-0e4692dbf0f1 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:08.101655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-d8016174-1fb7-4deb-ae7e-0e4692dbf0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:08.155334 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:08.163478 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1253/2510] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:52:08.183138 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152e499-4a7f-4a81-82ee-9b1d54aa6456 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:08.183430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152e499-4a7f-4a81-82ee-9b1d54aa6456 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:08.183712 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152e499-4a7f-4a81-82ee-9b1d54aa6456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:08.202315 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-3152e499-4a7f-4a81-82ee-9b1d54aa6456 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:08.203575 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-3152e499-4a7f-4a81-82ee-9b1d54aa6456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:08.212085 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1258/2511] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:08.216258 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8016174-1fb7-4deb-ae7e-0e4692dbf0f1 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:08.226212 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8016174-1fb7-4deb-ae7e-0e4692dbf0f1 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:08.231030 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291867b-683d-4daa-b70b-4c524d10d931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:08.231679 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291867b-683d-4daa-b70b-4c524d10d931 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:08.237686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291867b-683d-4daa-b70b-4c524d10d931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:08.237686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291867b-683d-4daa-b70b-4c524d10d931 admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:08.237686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9291867b-683d-4daa-b70b-4c524d10d931 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:52:08.237686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9291867b-683d-4daa-b70b-4c524d10d931 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:08.237686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9291867b-683d-4daa-b70b-4c524d10d931 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:08.248043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1254/2512] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:52:08.252607 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3152e499-4a7f-4a81-82ee-9b1d54aa6456 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:08.262510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3152e499-4a7f-4a81-82ee-9b1d54aa6456 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:08.267566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c04f5-3f41-44eb-9294-049b36c05e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:08.268454 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c04f5-3f41-44eb-9294-049b36c05e0b admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:08.268854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c04f5-3f41-44eb-9294-049b36c05e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:08.269287 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064c04f5-3f41-44eb-9294-049b36c05e0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:08.270301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064c04f5-3f41-44eb-9294-049b36c05e0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:08.273070 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064c04f5-3f41-44eb-9294-049b36c05e0b admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:08.353720 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1259/2513] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:52:08 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:08.357639 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291867b-683d-4daa-b70b-4c524d10d931 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:08.368792 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291867b-683d-4daa-b70b-4c524d10d931 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:08.424479 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac861c9-d70c-4451-a161-dc7544b0956a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:08.425131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac861c9-d70c-4451-a161-dc7544b0956a admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:08.425411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac861c9-d70c-4451-a161-dc7544b0956a admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:08.425685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac861c9-d70c-4451-a161-dc7544b0956a admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:08.426359 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac861c9-d70c-4451-a161-dc7544b0956a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:08.428292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac861c9-d70c-4451-a161-dc7544b0956a admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:08.506251 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1255/2514] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:52:08 2022] POST /identity/v3/users => generated 282 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:08.510340 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c04f5-3f41-44eb-9294-049b36c05e0b admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:08.520728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064c04f5-3f41-44eb-9294-049b36c05e0b admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:08.526979 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa74c8-782a-4f0f-adf7-1cdecaf17b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:08.527628 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa74c8-782a-4f0f-adf7-1cdecaf17b71 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:08.527907 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa74c8-782a-4f0f-adf7-1cdecaf17b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:08.528196 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa74c8-782a-4f0f-adf7-1cdecaf17b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:08.528894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa74c8-782a-4f0f-adf7-1cdecaf17b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:08.530516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa74c8-782a-4f0f-adf7-1cdecaf17b71 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:08.541752 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1260/2515] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:08.545772 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac861c9-d70c-4451-a161-dc7544b0956a admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:08.557396 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac861c9-d70c-4451-a161-dc7544b0956a admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:08.561894 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65173c62-d8c4-489a-95a6-d7c782497792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:08.562578 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65173c62-d8c4-489a-95a6-d7c782497792 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:08.562857 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65173c62-d8c4-489a-95a6-d7c782497792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:08.563155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65173c62-d8c4-489a-95a6-d7c782497792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:08.563870 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65173c62-d8c4-489a-95a6-d7c782497792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:08.565443 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65173c62-d8c4-489a-95a6-d7c782497792 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:08.575901 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1256/2516] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:52:08.580014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa74c8-782a-4f0f-adf7-1cdecaf17b71 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:08.589918 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa74c8-782a-4f0f-adf7-1cdecaf17b71 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:08.595239 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9a9de-e9ed-48f5-9f19-f8ff184c9b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:08.596270 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9a9de-e9ed-48f5-9f19-f8ff184c9b45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:08.596621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9a9de-e9ed-48f5-9f19-f8ff184c9b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:08.596941 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9a9de-e9ed-48f5-9f19-f8ff184c9b45 admin admin] PATH_INFO: `/v3/projects/4f4ae4fe234b420bbfc9a815893ccb91/users/40b362a49e6e4db18d4bcdfcb9d9cc68/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:08.613648 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f9a9de-e9ed-48f5-9f19-f8ff184c9b45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f4ae4fe234b420bbfc9a815893ccb91, user_id=40b362a49e6e4db18d4bcdfcb9d9cc68, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:08.616734 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f9a9de-e9ed-48f5-9f19-f8ff184c9b45 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:08.707475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1261/2517] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:08 2022] PUT /identity/v3/projects/4f4ae4fe234b420bbfc9a815893ccb91/users/40b362a49e6e4db18d4bcdfcb9d9cc68/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:08.724005 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c398d8-59d4-4e8a-b512-907a7a2f0282 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:08.724499 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c398d8-59d4-4e8a-b512-907a7a2f0282 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:08.724891 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c398d8-59d4-4e8a-b512-907a7a2f0282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:08.756506 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-a1c398d8-59d4-4e8a-b512-907a7a2f0282 None None] MFA Rules not processed for user `40b362a49e6e4db18d4bcdfcb9d9cc68`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:08.757744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-a1c398d8-59d4-4e8a-b512-907a7a2f0282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:08.825227 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1257/2518] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:08 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:08.829693 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9a9de-e9ed-48f5-9f19-f8ff184c9b45 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:08.834171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:08.893879 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9a9de-e9ed-48f5-9f19-f8ff184c9b45 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:08.899630 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d66d8-3174-4778-ab40-41d594442cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:08.900422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d66d8-3174-4778-ab40-41d594442cfa admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:08.900669 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d66d8-3174-4778-ab40-41d594442cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:08.901002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d66d8-3174-4778-ab40-41d594442cfa admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:08.901700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d66d8-3174-4778-ab40-41d594442cfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:08.903777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d66d8-3174-4778-ab40-41d594442cfa admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:08.988012 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1262/2519] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:52:08 2022] POST /identity/v3/projects => generated 365 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:08.992403 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c398d8-59d4-4e8a-b512-907a7a2f0282 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:09.003505 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c398d8-59d4-4e8a-b512-907a7a2f0282 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:09.059392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:09.066555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659b82f-96bf-4e09-9dd5-c6556699bd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:09.067223 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659b82f-96bf-4e09-9dd5-c6556699bd50 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:09.067508 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659b82f-96bf-4e09-9dd5-c6556699bd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:09.067785 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659b82f-96bf-4e09-9dd5-c6556699bd50 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:09.068574 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659b82f-96bf-4e09-9dd5-c6556699bd50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:09.070519 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659b82f-96bf-4e09-9dd5-c6556699bd50 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:09.164605 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1258/2520] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:52:08 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:09.169284 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d66d8-3174-4778-ab40-41d594442cfa admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:09.179771 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d66d8-3174-4778-ab40-41d594442cfa admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:09.184695 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a5f82-1b50-4b99-87f7-c9cd03e73def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:09.185358 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a5f82-1b50-4b99-87f7-c9cd03e73def admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:09.185643 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a5f82-1b50-4b99-87f7-c9cd03e73def admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:09.185930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a5f82-1b50-4b99-87f7-c9cd03e73def admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:09.186700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a5f82-1b50-4b99-87f7-c9cd03e73def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:09.188318 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a5f82-1b50-4b99-87f7-c9cd03e73def admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:09.201016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1263/2521] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:09.205332 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659b82f-96bf-4e09-9dd5-c6556699bd50 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:09.216230 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659b82f-96bf-4e09-9dd5-c6556699bd50 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:09.220959 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db4177-c536-4d70-87b7-2a9f4c8925d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:09.221837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db4177-c536-4d70-87b7-2a9f4c8925d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:09.222122 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db4177-c536-4d70-87b7-2a9f4c8925d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:09.222421 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db4177-c536-4d70-87b7-2a9f4c8925d4 admin admin] PATH_INFO: `/v3/projects/edba7ddfca16486daa289c6b4f74f05d/users/c15c1e5e429f48bd99ea695c42ee237a/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:09.237317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78db4177-c536-4d70-87b7-2a9f4c8925d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edba7ddfca16486daa289c6b4f74f05d, user_id=c15c1e5e429f48bd99ea695c42ee237a, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:09.240043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78db4177-c536-4d70-87b7-2a9f4c8925d4 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:09.330108 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1259/2522] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:09 2022] PUT /identity/v3/projects/edba7ddfca16486daa289c6b4f74f05d/users/c15c1e5e429f48bd99ea695c42ee237a/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:52:09.336073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a5f82-1b50-4b99-87f7-c9cd03e73def admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:09.347375 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a5f82-1b50-4b99-87f7-c9cd03e73def admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:09.404313 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2086db9-6db2-418c-8662-7211ed83794c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:09.404898 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2086db9-6db2-418c-8662-7211ed83794c admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:09.405180 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2086db9-6db2-418c-8662-7211ed83794c admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:09.405453 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2086db9-6db2-418c-8662-7211ed83794c admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:09.406171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2086db9-6db2-418c-8662-7211ed83794c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:09.407812 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2086db9-6db2-418c-8662-7211ed83794c admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:09.417734 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1264/2523] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:09.421922 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db4177-c536-4d70-87b7-2a9f4c8925d4 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:09.432916 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db4177-c536-4d70-87b7-2a9f4c8925d4 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:09.437507 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f562dec-a57e-415e-b364-c12a2a1cd706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:09.451362 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f562dec-a57e-415e-b364-c12a2a1cd706 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:09.451362 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f562dec-a57e-415e-b364-c12a2a1cd706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:09.451362 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f562dec-a57e-415e-b364-c12a2a1cd706 admin admin] PATH_INFO: `/v3/projects/edba7ddfca16486daa289c6b4f74f05d/users/c15c1e5e429f48bd99ea695c42ee237a/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:09.451362 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f562dec-a57e-415e-b364-c12a2a1cd706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edba7ddfca16486daa289c6b4f74f05d, user_id=c15c1e5e429f48bd99ea695c42ee237a, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:09.451362 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f562dec-a57e-415e-b364-c12a2a1cd706 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:09.454535 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1260/2524] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:09 2022] PUT /identity/v3/projects/edba7ddfca16486daa289c6b4f74f05d/users/c15c1e5e429f48bd99ea695c42ee237a/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:09.465414 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9beb5-aa08-481b-bc9d-7527d469bc69 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:09.465705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9beb5-aa08-481b-bc9d-7527d469bc69 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:09.465984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9beb5-aa08-481b-bc9d-7527d469bc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:09.491958 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-8fb9beb5-aa08-481b-bc9d-7527d469bc69 None None] MFA Rules not processed for user `c15c1e5e429f48bd99ea695c42ee237a`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:09.493244 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-8fb9beb5-aa08-481b-bc9d-7527d469bc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:09.556502 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:09.561656 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1265/2525] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:09 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:09.573511 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f562dec-a57e-415e-b364-c12a2a1cd706 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:09.575284 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-5f562dec-a57e-415e-b364-c12a2a1cd706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:09.633305 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f562dec-a57e-415e-b364-c12a2a1cd706 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:09.637161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e244f1-c093-4e8f-85e8-1052b6df9977 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:09.637797 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e244f1-c093-4e8f-85e8-1052b6df9977 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:09.638079 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e244f1-c093-4e8f-85e8-1052b6df9977 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:09.638354 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e244f1-c093-4e8f-85e8-1052b6df9977 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:09.647457 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e244f1-c093-4e8f-85e8-1052b6df9977 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:09.649368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e244f1-c093-4e8f-85e8-1052b6df9977 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:09.658891 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1261/2526] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:09 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:52:10.706905 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb9beb5-aa08-481b-bc9d-7527d469bc69 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:10.778410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb9beb5-aa08-481b-bc9d-7527d469bc69 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:10.783271 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a6aad-c3d2-41e7-b587-53e3aba8bbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:10.784443 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a6aad-c3d2-41e7-b587-53e3aba8bbe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:10.784443 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a6aad-c3d2-41e7-b587-53e3aba8bbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:10.784699 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a6aad-c3d2-41e7-b587-53e3aba8bbe4 admin admin] PATH_INFO: `/v3/users/40b362a49e6e4db18d4bcdfcb9d9cc68` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:10.786074 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a6aad-c3d2-41e7-b587-53e3aba8bbe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40b362a49e6e4db18d4bcdfcb9d9cc68)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:10.788141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a6aad-c3d2-41e7-b587-53e3aba8bbe4 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:10.944728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-1c3a6aad-c3d2-41e7-b587-53e3aba8bbe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40b362a49e6e4db18d4bcdfcb9d9cc68'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:10.952495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-1c3a6aad-c3d2-41e7-b587-53e3aba8bbe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40b362a49e6e4db18d4bcdfcb9d9cc68'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:10.953207 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-1c3a6aad-c3d2-41e7-b587-53e3aba8bbe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40b362a49e6e4db18d4bcdfcb9d9cc68'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:10.953819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-1c3a6aad-c3d2-41e7-b587-53e3aba8bbe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40b362a49e6e4db18d4bcdfcb9d9cc68'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:10.966161 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-1c3a6aad-c3d2-41e7-b587-53e3aba8bbe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40b362a49e6e4db18d4bcdfcb9d9cc68'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:11.039314 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1266/2527] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:52:10 2022] DELETE /identity/v3/users/40b362a49e6e4db18d4bcdfcb9d9cc68 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:11.045024 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f8fe7-c133-43fb-919a-eab741f69c0c None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:11.045024 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f8fe7-c133-43fb-919a-eab741f69c0c None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:11.045024 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f8fe7-c133-43fb-919a-eab741f69c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:11.064163 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-e10f8fe7-c133-43fb-919a-eab741f69c0c None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:11.065411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-e10f8fe7-c133-43fb-919a-eab741f69c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:11.133741 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1262/2528] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:52:11.143732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a6aad-c3d2-41e7-b587-53e3aba8bbe4 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:11.145180 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-1c3a6aad-c3d2-41e7-b587-53e3aba8bbe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:11.217915 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a6aad-c3d2-41e7-b587-53e3aba8bbe4 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:11.222585 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf536e-4c6c-4457-9259-0943c9117a3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:11.223236 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf536e-4c6c-4457-9259-0943c9117a3b service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:11.223520 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf536e-4c6c-4457-9259-0943c9117a3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:11.223845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf536e-4c6c-4457-9259-0943c9117a3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:11.234460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cf536e-4c6c-4457-9259-0943c9117a3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:11.236152 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cf536e-4c6c-4457-9259-0943c9117a3b service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:11.247447 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1267/2529] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:52:11.739208 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f8fe7-c133-43fb-919a-eab741f69c0c None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:11.743221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-e10f8fe7-c133-43fb-919a-eab741f69c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:11.758080 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f8fe7-c133-43fb-919a-eab741f69c0c None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:11.767759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a7e29-acec-4212-8b48-ce2979d8870b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:11.767759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a7e29-acec-4212-8b48-ce2979d8870b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:11.767759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a7e29-acec-4212-8b48-ce2979d8870b admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:11.767759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a7e29-acec-4212-8b48-ce2979d8870b admin admin] PATH_INFO: `/v3/projects/4f4ae4fe234b420bbfc9a815893ccb91` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:11.768158 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a7e29-acec-4212-8b48-ce2979d8870b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f4ae4fe234b420bbfc9a815893ccb91)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:11.768770 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a7e29-acec-4212-8b48-ce2979d8870b admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:11.947911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:11.951029 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-845a7e29-acec-4212-8b48-ce2979d8870b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f4ae4fe234b420bbfc9a815893ccb91', 'request_id': 'req-845a7e29-acec-4212-8b48-ce2979d8870b'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:11.993606 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-845a7e29-acec-4212-8b48-ce2979d8870b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f4ae4fe234b420bbfc9a815893ccb91', 'request_id': 'req-845a7e29-acec-4212-8b48-ce2979d8870b'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:12.004686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-845a7e29-acec-4212-8b48-ce2979d8870b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f4ae4fe234b420bbfc9a815893ccb91', 'request_id': 'req-845a7e29-acec-4212-8b48-ce2979d8870b'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:12.016687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-845a7e29-acec-4212-8b48-ce2979d8870b admin admin] The token cache is being invalidate because project 4f4ae4fe234b420bbfc9a815893ccb91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:52:12.017071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-845a7e29-acec-4212-8b48-ce2979d8870b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:12.017738 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-845a7e29-acec-4212-8b48-ce2979d8870b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:12.019355 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1263/2530] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:52:11 2022] DELETE /identity/v3/projects/4f4ae4fe234b420bbfc9a815893ccb91 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:12.023622 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf536e-4c6c-4457-9259-0943c9117a3b service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:12.033628 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf536e-4c6c-4457-9259-0943c9117a3b service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:12.050816 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:12.107623 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113e28e-70e7-477a-931e-c446c7f449d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:12.108512 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113e28e-70e7-477a-931e-c446c7f449d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:12.108792 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113e28e-70e7-477a-931e-c446c7f449d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:12.109066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113e28e-70e7-477a-931e-c446c7f449d5 admin admin] PATH_INFO: `/v3/users/c15c1e5e429f48bd99ea695c42ee237a` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:12.110472 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113e28e-70e7-477a-931e-c446c7f449d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c15c1e5e429f48bd99ea695c42ee237a)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:12.112646 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113e28e-70e7-477a-931e-c446c7f449d5 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:12.343736 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-5113e28e-70e7-477a-931e-c446c7f449d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c15c1e5e429f48bd99ea695c42ee237a'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:12.352963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-5113e28e-70e7-477a-931e-c446c7f449d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c15c1e5e429f48bd99ea695c42ee237a'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:12.353762 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-5113e28e-70e7-477a-931e-c446c7f449d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c15c1e5e429f48bd99ea695c42ee237a'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:12.354885 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-5113e28e-70e7-477a-931e-c446c7f449d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c15c1e5e429f48bd99ea695c42ee237a'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:12.381078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-5113e28e-70e7-477a-931e-c446c7f449d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c15c1e5e429f48bd99ea695c42ee237a'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:12.468397 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1268/2531] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:52:12 2022] DELETE /identity/v3/users/c15c1e5e429f48bd99ea695c42ee237a => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:12.909270 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a7e29-acec-4212-8b48-ce2979d8870b admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:12.910901 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-845a7e29-acec-4212-8b48-ce2979d8870b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:12.943998 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a7e29-acec-4212-8b48-ce2979d8870b admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:13.022202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522cb616-2863-4a48-95fc-b95aa6888896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:13.023104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cb616-2863-4a48-95fc-b95aa6888896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:13.023388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cb616-2863-4a48-95fc-b95aa6888896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:13.023666 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cb616-2863-4a48-95fc-b95aa6888896 admin admin] PATH_INFO: `/v3/projects/edba7ddfca16486daa289c6b4f74f05d` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:13.025914 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522cb616-2863-4a48-95fc-b95aa6888896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edba7ddfca16486daa289c6b4f74f05d)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:13.027862 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522cb616-2863-4a48-95fc-b95aa6888896 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:13.171908 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-522cb616-2863-4a48-95fc-b95aa6888896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edba7ddfca16486daa289c6b4f74f05d', 'request_id': 'req-522cb616-2863-4a48-95fc-b95aa6888896'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:13.264599 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-522cb616-2863-4a48-95fc-b95aa6888896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edba7ddfca16486daa289c6b4f74f05d', 'request_id': 'req-522cb616-2863-4a48-95fc-b95aa6888896'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:13.276642 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-522cb616-2863-4a48-95fc-b95aa6888896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edba7ddfca16486daa289c6b4f74f05d', 'request_id': 'req-522cb616-2863-4a48-95fc-b95aa6888896'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:13.291248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-522cb616-2863-4a48-95fc-b95aa6888896 admin admin] The token cache is being invalidate because project edba7ddfca16486daa289c6b4f74f05d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:52:13.291637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-522cb616-2863-4a48-95fc-b95aa6888896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:13.292529 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-522cb616-2863-4a48-95fc-b95aa6888896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:13.295248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1264/2532] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:52:12 2022] DELETE /identity/v3/projects/edba7ddfca16486daa289c6b4f74f05d => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:13.307154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201760a6-68fb-4651-b14d-358e5cfdd258 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:13.307154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201760a6-68fb-4651-b14d-358e5cfdd258 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:13.307154 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201760a6-68fb-4651-b14d-358e5cfdd258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:13.321862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-201760a6-68fb-4651-b14d-358e5cfdd258 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:13.323589 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-201760a6-68fb-4651-b14d-358e5cfdd258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:13.393124 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1269/2533] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:52:13.403097 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6013c-d30e-4768-993d-f3ca4d8bfc3d None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:13.403377 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6013c-d30e-4768-993d-f3ca4d8bfc3d None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:13.403820 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6013c-d30e-4768-993d-f3ca4d8bfc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:13.422593 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-c2a6013c-d30e-4768-993d-f3ca4d8bfc3d None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:13.423808 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-c2a6013c-d30e-4768-993d-f3ca4d8bfc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:13.431973 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1265/2534] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:13.436292 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201760a6-68fb-4651-b14d-358e5cfdd258 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:13.446483 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201760a6-68fb-4651-b14d-358e5cfdd258 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:13.451197 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f972aa-e3b3-4bbb-873f-32d58f334ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:13.451863 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f972aa-e3b3-4bbb-873f-32d58f334ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:13.452148 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f972aa-e3b3-4bbb-873f-32d58f334ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:13.452422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f972aa-e3b3-4bbb-873f-32d58f334ed5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:13.453156 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f972aa-e3b3-4bbb-873f-32d58f334ed5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:52:13.453463 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f972aa-e3b3-4bbb-873f-32d58f334ed5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:13.455136 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f972aa-e3b3-4bbb-873f-32d58f334ed5 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:13.468434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1270/2535] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:52:13.472779 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6013c-d30e-4768-993d-f3ca4d8bfc3d None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:13.483661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6013c-d30e-4768-993d-f3ca4d8bfc3d None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:13.488924 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d3a6e-8d91-4dad-97a1-f1d54e01f118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:13.489570 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d3a6e-8d91-4dad-97a1-f1d54e01f118 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:13.489850 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d3a6e-8d91-4dad-97a1-f1d54e01f118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:13.490125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d3a6e-8d91-4dad-97a1-f1d54e01f118 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:13.490917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d3a6e-8d91-4dad-97a1-f1d54e01f118 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:13.493145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d3a6e-8d91-4dad-97a1-f1d54e01f118 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:13.575483 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f972aa-e3b3-4bbb-873f-32d58f334ed5 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:13.578554 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-81f972aa-e3b3-4bbb-873f-32d58f334ed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:13.591140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f972aa-e3b3-4bbb-873f-32d58f334ed5 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:13.595890 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd439d-201f-40c6-b0ef-4c3ab72f07f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:13.596780 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd439d-201f-40c6-b0ef-4c3ab72f07f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:13.597065 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd439d-201f-40c6-b0ef-4c3ab72f07f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:13.597348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd439d-201f-40c6-b0ef-4c3ab72f07f2 admin admin] PATH_INFO: `/v3/users/f7fd83a789aa4a5b87c1bdfdc3e20da3` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:13.598586 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd439d-201f-40c6-b0ef-4c3ab72f07f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7fd83a789aa4a5b87c1bdfdc3e20da3)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:13.600520 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd439d-201f-40c6-b0ef-4c3ab72f07f2 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:13.607232 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1266/2536] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:52:13 2022] POST /identity/v3/projects => generated 363 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:13.611692 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d3a6e-8d91-4dad-97a1-f1d54e01f118 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:13.629449 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:13.655471 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:13.678968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d3a6e-8d91-4dad-97a1-f1d54e01f118 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:13.685095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32080d6-31d8-4e43-b88b-8d48b1454890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:13.685530 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32080d6-31d8-4e43-b88b-8d48b1454890 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:13.685816 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32080d6-31d8-4e43-b88b-8d48b1454890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:13.686092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32080d6-31d8-4e43-b88b-8d48b1454890 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:13.686825 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32080d6-31d8-4e43-b88b-8d48b1454890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:13.688844 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32080d6-31d8-4e43-b88b-8d48b1454890 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:13.790824 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1267/2537] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:52:13 2022] POST /identity/v3/users => generated 282 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:52:13.793485 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32080d6-31d8-4e43-b88b-8d48b1454890 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:13.804257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32080d6-31d8-4e43-b88b-8d48b1454890 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:13.809804 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec41e8-9bbc-4e6e-99e6-e27d6e7bc612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:13.811127 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec41e8-9bbc-4e6e-99e6-e27d6e7bc612 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:13.811390 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec41e8-9bbc-4e6e-99e6-e27d6e7bc612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:13.822061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec41e8-9bbc-4e6e-99e6-e27d6e7bc612 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:13.822964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec41e8-9bbc-4e6e-99e6-e27d6e7bc612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:13.824625 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec41e8-9bbc-4e6e-99e6-e27d6e7bc612 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:13.833430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-dddd439d-201f-40c6-b0ef-4c3ab72f07f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7fd83a789aa4a5b87c1bdfdc3e20da3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:13.837897 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1268/2538] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:13.847690 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec41e8-9bbc-4e6e-99e6-e27d6e7bc612 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:13.847787 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-dddd439d-201f-40c6-b0ef-4c3ab72f07f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7fd83a789aa4a5b87c1bdfdc3e20da3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:13.847787 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-dddd439d-201f-40c6-b0ef-4c3ab72f07f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7fd83a789aa4a5b87c1bdfdc3e20da3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:13.847787 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-dddd439d-201f-40c6-b0ef-4c3ab72f07f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7fd83a789aa4a5b87c1bdfdc3e20da3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:13.852946 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec41e8-9bbc-4e6e-99e6-e27d6e7bc612 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:13.857633 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-dddd439d-201f-40c6-b0ef-4c3ab72f07f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7fd83a789aa4a5b87c1bdfdc3e20da3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:13.858036 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5789c44c-7454-40da-8940-b5a31157b000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:13.858855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789c44c-7454-40da-8940-b5a31157b000 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:13.859139 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789c44c-7454-40da-8940-b5a31157b000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:13.859420 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5789c44c-7454-40da-8940-b5a31157b000 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:13.860145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5789c44c-7454-40da-8940-b5a31157b000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:13.861739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5789c44c-7454-40da-8940-b5a31157b000 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:13.872418 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1269/2539] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:13.876906 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5789c44c-7454-40da-8940-b5a31157b000 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:13.878600 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-5789c44c-7454-40da-8940-b5a31157b000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:13.891647 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5789c44c-7454-40da-8940-b5a31157b000 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:13.897024 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca6a35-f610-4a96-b374-372c8eadef74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:13.897871 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca6a35-f610-4a96-b374-372c8eadef74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:13.898155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca6a35-f610-4a96-b374-372c8eadef74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:13.898461 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca6a35-f610-4a96-b374-372c8eadef74 admin admin] PATH_INFO: `/v3/projects/c6355ed0a51243d283a1092855634f7f/users/46f6212ba6f84b70b5944686c8f31def/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:13.913757 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca6a35-f610-4a96-b374-372c8eadef74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6355ed0a51243d283a1092855634f7f, user_id=46f6212ba6f84b70b5944686c8f31def, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:13.916515 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca6a35-f610-4a96-b374-372c8eadef74 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:13.929966 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1271/2540] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:52:13 2022] DELETE /identity/v3/users/f7fd83a789aa4a5b87c1bdfdc3e20da3 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:13.935352 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad0d56-f891-447f-a488-7ecaa84ef7e5 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:13.935716 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad0d56-f891-447f-a488-7ecaa84ef7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:13.936064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad0d56-f891-447f-a488-7ecaa84ef7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:13.955028 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-16ad0d56-f891-447f-a488-7ecaa84ef7e5 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:13.956276 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-16ad0d56-f891-447f-a488-7ecaa84ef7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:14.011129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1270/2541] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:13 2022] PUT /identity/v3/projects/c6355ed0a51243d283a1092855634f7f/users/46f6212ba6f84b70b5944686c8f31def/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:52:14.017685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1272/2542] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:14.021472 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa490a03-6d27-441e-b40a-18d33f544d63 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:14.021830 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa490a03-6d27-441e-b40a-18d33f544d63 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:14.022640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa490a03-6d27-441e-b40a-18d33f544d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:14.028015 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad0d56-f891-447f-a488-7ecaa84ef7e5 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:14.029349 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-16ad0d56-f891-447f-a488-7ecaa84ef7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:14.056705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-aa490a03-6d27-441e-b40a-18d33f544d63 None None] MFA Rules not processed for user `46f6212ba6f84b70b5944686c8f31def`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:14.057624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-aa490a03-6d27-441e-b40a-18d33f544d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:14.101733 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad0d56-f891-447f-a488-7ecaa84ef7e5 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:14.125169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1271/2543] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:14 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:52:14.129553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa490a03-6d27-441e-b40a-18d33f544d63 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:14.152784 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e29e5a-3d3f-4bf7-bf1d-cf0fef71afec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:14.152956 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e29e5a-3d3f-4bf7-bf1d-cf0fef71afec service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:14.153574 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e29e5a-3d3f-4bf7-bf1d-cf0fef71afec service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:14.154796 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e29e5a-3d3f-4bf7-bf1d-cf0fef71afec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:14.200414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa490a03-6d27-441e-b40a-18d33f544d63 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:14.205166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38f64a-22f5-4b1e-aa26-3a63f5ff81ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:14.206110 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38f64a-22f5-4b1e-aa26-3a63f5ff81ef admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:14.206453 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38f64a-22f5-4b1e-aa26-3a63f5ff81ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:14.206705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38f64a-22f5-4b1e-aa26-3a63f5ff81ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:14.207411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38f64a-22f5-4b1e-aa26-3a63f5ff81ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:14.209403 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38f64a-22f5-4b1e-aa26-3a63f5ff81ef admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:14.226047 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e29e5a-3d3f-4bf7-bf1d-cf0fef71afec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:14.227886 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e29e5a-3d3f-4bf7-bf1d-cf0fef71afec service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:14.239010 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1273/2544] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:52:14.292288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1272/2545] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:52:14 2022] POST /identity/v3/projects => generated 365 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:52:14.296275 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e29e5a-3d3f-4bf7-bf1d-cf0fef71afec service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:14.306757 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e29e5a-3d3f-4bf7-bf1d-cf0fef71afec service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:14.314340 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:14.370318 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691803f-098f-43d7-8dfd-b7fab1d93a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:14.371024 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691803f-098f-43d7-8dfd-b7fab1d93a79 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:14.371306 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691803f-098f-43d7-8dfd-b7fab1d93a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:14.371580 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691803f-098f-43d7-8dfd-b7fab1d93a79 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:14.372276 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691803f-098f-43d7-8dfd-b7fab1d93a79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:14.374302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691803f-098f-43d7-8dfd-b7fab1d93a79 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:14.513603 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:14.523471 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1274/2546] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:52:14 2022] POST /identity/v3/users => generated 289 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:14.526720 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38f64a-22f5-4b1e-aa26-3a63f5ff81ef admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:14.538771 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38f64a-22f5-4b1e-aa26-3a63f5ff81ef admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:14.544475 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73766a60-b93d-420f-8c63-9ee96d48131e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:14.545740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73766a60-b93d-420f-8c63-9ee96d48131e admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:14.545740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73766a60-b93d-420f-8c63-9ee96d48131e admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:14.545976 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73766a60-b93d-420f-8c63-9ee96d48131e admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:14.546762 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73766a60-b93d-420f-8c63-9ee96d48131e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:14.550868 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73766a60-b93d-420f-8c63-9ee96d48131e admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:14.562880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1273/2547] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:14.566853 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691803f-098f-43d7-8dfd-b7fab1d93a79 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:14.577830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691803f-098f-43d7-8dfd-b7fab1d93a79 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:14.582907 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8eb5b-fd8b-4024-8390-5095200c36ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:14.583881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8eb5b-fd8b-4024-8390-5095200c36ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:14.584257 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8eb5b-fd8b-4024-8390-5095200c36ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:14.584610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8eb5b-fd8b-4024-8390-5095200c36ef admin admin] PATH_INFO: `/v3/projects/3f2b66a8827048619123cfb55483d869/users/9227a6272fd54efa889097ec795fa088/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:14.601491 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b8eb5b-fd8b-4024-8390-5095200c36ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f2b66a8827048619123cfb55483d869, user_id=9227a6272fd54efa889097ec795fa088, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:14.605090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b8eb5b-fd8b-4024-8390-5095200c36ef admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:14.690405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1275/2548] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:14 2022] PUT /identity/v3/projects/3f2b66a8827048619123cfb55483d869/users/9227a6272fd54efa889097ec795fa088/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:52:14.694147 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73766a60-b93d-420f-8c63-9ee96d48131e admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:14.704772 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73766a60-b93d-420f-8c63-9ee96d48131e admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:14.709887 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8eb5b-fd8b-4024-8390-5095200c36ef admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:14.711695 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-17b8eb5b-fd8b-4024-8390-5095200c36ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:14.792095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f6c37-b875-4d6c-98d0-a8fb98a11ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:14.792763 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f6c37-b875-4d6c-98d0-a8fb98a11ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:14.793048 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f6c37-b875-4d6c-98d0-a8fb98a11ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:14.793326 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f6c37-b875-4d6c-98d0-a8fb98a11ee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:14.794051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f6c37-b875-4d6c-98d0-a8fb98a11ee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:14.795726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f6c37-b875-4d6c-98d0-a8fb98a11ee4 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:14.796508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8eb5b-fd8b-4024-8390-5095200c36ef admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:14.801198 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41bf51-6c71-4b37-a77e-34f3dc899581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:14.802056 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41bf51-6c71-4b37-a77e-34f3dc899581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:14.802334 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41bf51-6c71-4b37-a77e-34f3dc899581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:14.802798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41bf51-6c71-4b37-a77e-34f3dc899581 admin admin] PATH_INFO: `/v3/projects/857fa274a1594a258bb662e743776f46` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:14.804150 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41bf51-6c71-4b37-a77e-34f3dc899581 admin admin] RBAC: Authorizing `identity:delete_project(project_id=857fa274a1594a258bb662e743776f46)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:14.806100 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1274/2549] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:14.809825 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41bf51-6c71-4b37-a77e-34f3dc899581 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:14.810254 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f6c37-b875-4d6c-98d0-a8fb98a11ee4 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:14.819824 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f6c37-b875-4d6c-98d0-a8fb98a11ee4 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:14.824566 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad7b9d-cdb5-45c8-8fc0-f6ccce08a9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:14.825422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad7b9d-cdb5-45c8-8fc0-f6ccce08a9df admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:14.825701 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad7b9d-cdb5-45c8-8fc0-f6ccce08a9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:14.825978 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad7b9d-cdb5-45c8-8fc0-f6ccce08a9df admin admin] PATH_INFO: `/v3/projects/3f2b66a8827048619123cfb55483d869/users/9227a6272fd54efa889097ec795fa088/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:14.828007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad7b9d-cdb5-45c8-8fc0-f6ccce08a9df admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f2b66a8827048619123cfb55483d869, user_id=9227a6272fd54efa889097ec795fa088, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:14.830680 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad7b9d-cdb5-45c8-8fc0-f6ccce08a9df admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:14.842349 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1275/2550] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:14 2022] PUT /identity/v3/projects/3f2b66a8827048619123cfb55483d869/users/9227a6272fd54efa889097ec795fa088/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:52:14.853453 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730bbac-5de9-4049-9abb-b22fbf9e71ea None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:14.853453 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730bbac-5de9-4049-9abb-b22fbf9e71ea None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:14.853453 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730bbac-5de9-4049-9abb-b22fbf9e71ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:14.881201 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-e730bbac-5de9-4049-9abb-b22fbf9e71ea None None] MFA Rules not processed for user `9227a6272fd54efa889097ec795fa088`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:14.882778 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-e730bbac-5de9-4049-9abb-b22fbf9e71ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:14.909346 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-8d41bf51-6c71-4b37-a77e-34f3dc899581 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '857fa274a1594a258bb662e743776f46', 'request_id': 'req-8d41bf51-6c71-4b37-a77e-34f3dc899581'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:14.959113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1276/2551] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:14 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:14.971471 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730bbac-5de9-4049-9abb-b22fbf9e71ea None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:14.972851 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-e730bbac-5de9-4049-9abb-b22fbf9e71ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:15.011030 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-8d41bf51-6c71-4b37-a77e-34f3dc899581 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '857fa274a1594a258bb662e743776f46', 'request_id': 'req-8d41bf51-6c71-4b37-a77e-34f3dc899581'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:15.017775 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-8d41bf51-6c71-4b37-a77e-34f3dc899581 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '857fa274a1594a258bb662e743776f46', 'request_id': 'req-8d41bf51-6c71-4b37-a77e-34f3dc899581'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:15.023182 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:15.031604 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-8d41bf51-6c71-4b37-a77e-34f3dc899581 admin admin] The token cache is being invalidate because project 857fa274a1594a258bb662e743776f46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:52:15.032089 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-8d41bf51-6c71-4b37-a77e-34f3dc899581 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:15.033456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-8d41bf51-6c71-4b37-a77e-34f3dc899581 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:15.034261 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730bbac-5de9-4049-9abb-b22fbf9e71ea None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:15.038699 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1276/2552] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:52:14 2022] DELETE /identity/v3/projects/857fa274a1594a258bb662e743776f46 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:15.049268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:15.049415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41bf51-6c71-4b37-a77e-34f3dc899581 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:15.049415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-8d41bf51-6c71-4b37-a77e-34f3dc899581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:15.091435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399cb4c-a2ce-4fac-b442-eb3d0b516d34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:15.092092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399cb4c-a2ce-4fac-b442-eb3d0b516d34 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:15.092374 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399cb4c-a2ce-4fac-b442-eb3d0b516d34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:15.092649 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399cb4c-a2ce-4fac-b442-eb3d0b516d34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:15.095847 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f399cb4c-a2ce-4fac-b442-eb3d0b516d34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:15.120708 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41bf51-6c71-4b37-a77e-34f3dc899581 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:15.126667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398723df-349a-495e-8b0c-ef61c4a338a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:15.127829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398723df-349a-495e-8b0c-ef61c4a338a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:15.128246 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398723df-349a-495e-8b0c-ef61c4a338a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:15.128630 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398723df-349a-495e-8b0c-ef61c4a338a8 admin admin] PATH_INFO: `/v3/users/8bb2911660254e3f89326ef680e0e4d0` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:15.129916 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398723df-349a-495e-8b0c-ef61c4a338a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bb2911660254e3f89326ef680e0e4d0)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:15.132166 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398723df-349a-495e-8b0c-ef61c4a338a8 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:15.183491 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399cb4c-a2ce-4fac-b442-eb3d0b516d34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:15.185073 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399cb4c-a2ce-4fac-b442-eb3d0b516d34 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:15.201127 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1277/2553] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:14 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:52:15.276324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-398723df-349a-495e-8b0c-ef61c4a338a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bb2911660254e3f89326ef680e0e4d0'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:15.283839 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-398723df-349a-495e-8b0c-ef61c4a338a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bb2911660254e3f89326ef680e0e4d0'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:15.284554 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-398723df-349a-495e-8b0c-ef61c4a338a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bb2911660254e3f89326ef680e0e4d0'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:15.285176 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-398723df-349a-495e-8b0c-ef61c4a338a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bb2911660254e3f89326ef680e0e4d0'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:15.300777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-398723df-349a-495e-8b0c-ef61c4a338a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bb2911660254e3f89326ef680e0e4d0'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:15.405035 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1277/2554] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:52:15 2022] DELETE /identity/v3/users/8bb2911660254e3f89326ef680e0e4d0 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:15.416481 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399cb4c-a2ce-4fac-b442-eb3d0b516d34 service cinder] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:15.417816 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f399cb4c-a2ce-4fac-b442-eb3d0b516d34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:15.434435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399cb4c-a2ce-4fac-b442-eb3d0b516d34 service cinder] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:15.501441 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec8792-0c1c-49eb-a66b-edece8eadbab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:15.502184 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec8792-0c1c-49eb-a66b-edece8eadbab service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:15.502626 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec8792-0c1c-49eb-a66b-edece8eadbab service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:15.503002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec8792-0c1c-49eb-a66b-edece8eadbab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:15.504904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-ceec8792-0c1c-49eb-a66b-edece8eadbab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:15.566577 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec8792-0c1c-49eb-a66b-edece8eadbab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:15.567923 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec8792-0c1c-49eb-a66b-edece8eadbab service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:15.577336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1278/2555] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:15 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:52:16.074524 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398723df-349a-495e-8b0c-ef61c4a338a8 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:16.076014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-398723df-349a-495e-8b0c-ef61c4a338a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:16.100103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:16.143459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398723df-349a-495e-8b0c-ef61c4a338a8 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:16.148785 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46f9c6-1d14-497a-abdb-3a4c31adee3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:16.149643 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46f9c6-1d14-497a-abdb-3a4c31adee3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:16.149924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46f9c6-1d14-497a-abdb-3a4c31adee3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:16.150196 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46f9c6-1d14-497a-abdb-3a4c31adee3b admin admin] PATH_INFO: `/v3/projects/2f666ce0759440168580334451f7e377` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:16.151466 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46f9c6-1d14-497a-abdb-3a4c31adee3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f666ce0759440168580334451f7e377)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:16.153515 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46f9c6-1d14-497a-abdb-3a4c31adee3b admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:16.284234 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-df46f9c6-1d14-497a-abdb-3a4c31adee3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f666ce0759440168580334451f7e377', 'request_id': 'req-df46f9c6-1d14-497a-abdb-3a4c31adee3b'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:16.358408 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-df46f9c6-1d14-497a-abdb-3a4c31adee3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f666ce0759440168580334451f7e377', 'request_id': 'req-df46f9c6-1d14-497a-abdb-3a4c31adee3b'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:16.365197 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-df46f9c6-1d14-497a-abdb-3a4c31adee3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f666ce0759440168580334451f7e377', 'request_id': 'req-df46f9c6-1d14-497a-abdb-3a4c31adee3b'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:16.376341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-df46f9c6-1d14-497a-abdb-3a4c31adee3b admin admin] The token cache is being invalidate because project 2f666ce0759440168580334451f7e377 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:52:16.376629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-df46f9c6-1d14-497a-abdb-3a4c31adee3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:16.377258 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-df46f9c6-1d14-497a-abdb-3a4c31adee3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:16.378866 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1278/2556] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:52:16 2022] DELETE /identity/v3/projects/2f666ce0759440168580334451f7e377 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:16.384889 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b66f7d-af60-4e79-bce3-8d84cd5daedd None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:16.385491 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b66f7d-af60-4e79-bce3-8d84cd5daedd None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:16.387400 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b66f7d-af60-4e79-bce3-8d84cd5daedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:16.405052 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-55b66f7d-af60-4e79-bce3-8d84cd5daedd None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:16.406252 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-55b66f7d-af60-4e79-bce3-8d84cd5daedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:16.411775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:16.469441 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1279/2557] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:16.479174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15139cc1-1264-4406-9fac-32803702d16c None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:16.479635 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15139cc1-1264-4406-9fac-32803702d16c None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:16.479724 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15139cc1-1264-4406-9fac-32803702d16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:16.500173 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-15139cc1-1264-4406-9fac-32803702d16c None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:16.501765 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-15139cc1-1264-4406-9fac-32803702d16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:16.509563 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1279/2558] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:16.514040 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b66f7d-af60-4e79-bce3-8d84cd5daedd None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:16.524652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b66f7d-af60-4e79-bce3-8d84cd5daedd None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:16.533317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b5cb3-0100-42b8-a756-aacbb5c20792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:16.533984 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b5cb3-0100-42b8-a756-aacbb5c20792 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:16.534265 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b5cb3-0100-42b8-a756-aacbb5c20792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:16.534572 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b5cb3-0100-42b8-a756-aacbb5c20792 admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:16.535308 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113b5cb3-0100-42b8-a756-aacbb5c20792 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:52:16.535615 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113b5cb3-0100-42b8-a756-aacbb5c20792 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:16.537277 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113b5cb3-0100-42b8-a756-aacbb5c20792 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:16.538578 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15139cc1-1264-4406-9fac-32803702d16c None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:16.540657 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-15139cc1-1264-4406-9fac-32803702d16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:16.550414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1280/2559] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:52:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:52:16.554582 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b5cb3-0100-42b8-a756-aacbb5c20792 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:16.564323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b5cb3-0100-42b8-a756-aacbb5c20792 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:16.568962 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54890567-d031-4e9c-9b23-3216544219b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:16.569582 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54890567-d031-4e9c-9b23-3216544219b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:16.569863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54890567-d031-4e9c-9b23-3216544219b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:16.570137 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54890567-d031-4e9c-9b23-3216544219b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:16.570891 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54890567-d031-4e9c-9b23-3216544219b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:16.572931 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54890567-d031-4e9c-9b23-3216544219b4 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:16.598245 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15139cc1-1264-4406-9fac-32803702d16c None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:16.603539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd9687-498a-4d8a-bb75-354adddd614e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:16.603539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd9687-498a-4d8a-bb75-354adddd614e service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:16.603778 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd9687-498a-4d8a-bb75-354adddd614e service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:16.603875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd9687-498a-4d8a-bb75-354adddd614e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:16.605640 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-b0cd9687-498a-4d8a-bb75-354adddd614e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:16.617811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd9687-498a-4d8a-bb75-354adddd614e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:16.619247 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd9687-498a-4d8a-bb75-354adddd614e service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:16.628852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1280/2560] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:52:16.674655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1281/2561] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:52:16 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:16.678882 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd9687-498a-4d8a-bb75-354adddd614e service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:16.688681 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd9687-498a-4d8a-bb75-354adddd614e service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:16.732641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:16.745587 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73886ea-0696-47e7-9aeb-965f92b4f246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:16.746354 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73886ea-0696-47e7-9aeb-965f92b4f246 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:16.746728 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73886ea-0696-47e7-9aeb-965f92b4f246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:16.747077 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73886ea-0696-47e7-9aeb-965f92b4f246 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:16.747821 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73886ea-0696-47e7-9aeb-965f92b4f246 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:16.750045 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73886ea-0696-47e7-9aeb-965f92b4f246 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:16.827770 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1281/2562] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:52:16 2022] POST /identity/v3/users => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:16.832028 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54890567-d031-4e9c-9b23-3216544219b4 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:16.843072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54890567-d031-4e9c-9b23-3216544219b4 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:16.847649 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa606804-d334-4b91-9052-1215ee939ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:16.848344 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa606804-d334-4b91-9052-1215ee939ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:16.848598 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa606804-d334-4b91-9052-1215ee939ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:16.848855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa606804-d334-4b91-9052-1215ee939ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:16.851531 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa606804-d334-4b91-9052-1215ee939ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:16.851531 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa606804-d334-4b91-9052-1215ee939ce2 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:16.864481 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1282/2563] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:52:16.865957 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73886ea-0696-47e7-9aeb-965f92b4f246 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:16.876963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73886ea-0696-47e7-9aeb-965f92b4f246 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:16.881732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77eb25a-e575-4bd1-8971-d2008c9f81aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:16.882595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eb25a-e575-4bd1-8971-d2008c9f81aa admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:16.882949 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eb25a-e575-4bd1-8971-d2008c9f81aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:16.883298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eb25a-e575-4bd1-8971-d2008c9f81aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:16.884600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77eb25a-e575-4bd1-8971-d2008c9f81aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:16.886467 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77eb25a-e575-4bd1-8971-d2008c9f81aa admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:16.897274 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1282/2564] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:16.902213 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa606804-d334-4b91-9052-1215ee939ce2 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:16.912398 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa606804-d334-4b91-9052-1215ee939ce2 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:16.917017 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b599cd-7cf9-4e62-9163-10f7a7a76386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:16.917882 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b599cd-7cf9-4e62-9163-10f7a7a76386 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:16.918156 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b599cd-7cf9-4e62-9163-10f7a7a76386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:16.918446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b599cd-7cf9-4e62-9163-10f7a7a76386 admin admin] PATH_INFO: `/v3/projects/ac1847cdd8cc43d9b34490bd5e871e2b/users/aa14260afbce48ffbb851ff7065aea64/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:16.936352 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b599cd-7cf9-4e62-9163-10f7a7a76386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac1847cdd8cc43d9b34490bd5e871e2b, user_id=aa14260afbce48ffbb851ff7065aea64, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:16.939175 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b599cd-7cf9-4e62-9163-10f7a7a76386 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:17.004746 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1283/2565] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:16 2022] PUT /identity/v3/projects/ac1847cdd8cc43d9b34490bd5e871e2b/users/aa14260afbce48ffbb851ff7065aea64/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:52:17.015978 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec562d21-487b-45ef-9233-0b414759c8b5 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:17.016297 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec562d21-487b-45ef-9233-0b414759c8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:17.016576 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec562d21-487b-45ef-9233-0b414759c8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:17.044365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-ec562d21-487b-45ef-9233-0b414759c8b5 None None] MFA Rules not processed for user `aa14260afbce48ffbb851ff7065aea64`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:17.045677 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ec562d21-487b-45ef-9233-0b414759c8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:17.125329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1283/2566] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:17.136733 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b599cd-7cf9-4e62-9163-10f7a7a76386 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:17.138628 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-d7b599cd-7cf9-4e62-9163-10f7a7a76386 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:17.203021 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b599cd-7cf9-4e62-9163-10f7a7a76386 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:17.206957 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66239e96-730a-4eab-ba5d-2b73134036fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:17.207605 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66239e96-730a-4eab-ba5d-2b73134036fd service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:17.207890 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66239e96-730a-4eab-ba5d-2b73134036fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:17.208171 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66239e96-730a-4eab-ba5d-2b73134036fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:17.222872 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66239e96-730a-4eab-ba5d-2b73134036fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:17.228002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66239e96-730a-4eab-ba5d-2b73134036fd service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:17.238147 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1284/2567] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:52:19.055196 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70ebfd-46ce-4f6f-986e-e756b9e5a492 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:19.055780 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70ebfd-46ce-4f6f-986e-e756b9e5a492 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:19.059023 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70ebfd-46ce-4f6f-986e-e756b9e5a492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:19.074539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-de70ebfd-46ce-4f6f-986e-e756b9e5a492 None None] MFA Rules not processed for user `aa14260afbce48ffbb851ff7065aea64`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:19.075817 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-de70ebfd-46ce-4f6f-986e-e756b9e5a492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:19.083303 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1284/2568] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:20.918733 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66239e96-730a-4eab-ba5d-2b73134036fd service cinder] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:20.920246 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-66239e96-730a-4eab-ba5d-2b73134036fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:20.971076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:20.990903 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66239e96-730a-4eab-ba5d-2b73134036fd service cinder] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:20.995823 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c07da-d9ba-4a37-8d6a-d9e63fc4a364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:20.996688 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c07da-d9ba-4a37-8d6a-d9e63fc4a364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:20.997057 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c07da-d9ba-4a37-8d6a-d9e63fc4a364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:20.997281 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c07da-d9ba-4a37-8d6a-d9e63fc4a364 admin admin] PATH_INFO: `/v3/users/f6d74ace9f674f0da40cb35f39d559c9` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:20.998501 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c07da-d9ba-4a37-8d6a-d9e63fc4a364 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6d74ace9f674f0da40cb35f39d559c9)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:21.000406 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c07da-d9ba-4a37-8d6a-d9e63fc4a364 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:21.139415 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-294c07da-d9ba-4a37-8d6a-d9e63fc4a364 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6d74ace9f674f0da40cb35f39d559c9'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:21.148652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-294c07da-d9ba-4a37-8d6a-d9e63fc4a364 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6d74ace9f674f0da40cb35f39d559c9'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:21.149925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-294c07da-d9ba-4a37-8d6a-d9e63fc4a364 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6d74ace9f674f0da40cb35f39d559c9'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:21.162535 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-294c07da-d9ba-4a37-8d6a-d9e63fc4a364 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6d74ace9f674f0da40cb35f39d559c9'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:21.227156 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-294c07da-d9ba-4a37-8d6a-d9e63fc4a364 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6d74ace9f674f0da40cb35f39d559c9'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:21.233989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1285/2569] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:52:20 2022] DELETE /identity/v3/users/f6d74ace9f674f0da40cb35f39d559c9 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:21.490496 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70ebfd-46ce-4f6f-986e-e756b9e5a492 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:21.504130 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70ebfd-46ce-4f6f-986e-e756b9e5a492 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:21.563159 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f46af-2848-4e79-817b-b71d8f6d77f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:21.564018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f46af-2848-4e79-817b-b71d8f6d77f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:21.564305 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f46af-2848-4e79-817b-b71d8f6d77f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:21.564581 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f46af-2848-4e79-817b-b71d8f6d77f3 admin admin] PATH_INFO: `/v3/projects/80ec901270bc4375b22fba22ed2ff68c` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:21.565756 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f46af-2848-4e79-817b-b71d8f6d77f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80ec901270bc4375b22fba22ed2ff68c)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:21.567739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f46af-2848-4e79-817b-b71d8f6d77f3 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:21.661528 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b23f46af-2848-4e79-817b-b71d8f6d77f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80ec901270bc4375b22fba22ed2ff68c', 'request_id': 'req-b23f46af-2848-4e79-817b-b71d8f6d77f3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:21.663181 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:21.718295 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b23f46af-2848-4e79-817b-b71d8f6d77f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80ec901270bc4375b22fba22ed2ff68c', 'request_id': 'req-b23f46af-2848-4e79-817b-b71d8f6d77f3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:21.724819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b23f46af-2848-4e79-817b-b71d8f6d77f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80ec901270bc4375b22fba22ed2ff68c', 'request_id': 'req-b23f46af-2848-4e79-817b-b71d8f6d77f3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:21.735871 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b23f46af-2848-4e79-817b-b71d8f6d77f3 admin admin] The token cache is being invalidate because project 80ec901270bc4375b22fba22ed2ff68c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:52:21.736410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b23f46af-2848-4e79-817b-b71d8f6d77f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:21.737384 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b23f46af-2848-4e79-817b-b71d8f6d77f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:21.739670 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1285/2570] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:52:21 2022] DELETE /identity/v3/projects/80ec901270bc4375b22fba22ed2ff68c => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:21.744783 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1822a95-ad56-43e0-aad7-bf8f74a94d59 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:21.745071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1822a95-ad56-43e0-aad7-bf8f74a94d59 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:21.745353 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1822a95-ad56-43e0-aad7-bf8f74a94d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:21.765627 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-c1822a95-ad56-43e0-aad7-bf8f74a94d59 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:21.766036 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-c1822a95-ad56-43e0-aad7-bf8f74a94d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:21.829079 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1286/2571] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:21.838225 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d4926-05b2-4442-85f8-4fb3575ac2cb None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:21.838543 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d4926-05b2-4442-85f8-4fb3575ac2cb None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:21.838820 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d4926-05b2-4442-85f8-4fb3575ac2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:21.859851 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-d70d4926-05b2-4442-85f8-4fb3575ac2cb None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:21.860819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-d70d4926-05b2-4442-85f8-4fb3575ac2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:21.870650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1286/2572] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:52:21.874944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1822a95-ad56-43e0-aad7-bf8f74a94d59 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:21.885617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1822a95-ad56-43e0-aad7-bf8f74a94d59 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:21.890496 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d93ff8-353d-4428-a46e-64c9b0f4df27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:21.891148 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d93ff8-353d-4428-a46e-64c9b0f4df27 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:21.891433 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d93ff8-353d-4428-a46e-64c9b0f4df27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:21.891842 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d93ff8-353d-4428-a46e-64c9b0f4df27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:21.892573 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d93ff8-353d-4428-a46e-64c9b0f4df27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:52:21.892887 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d93ff8-353d-4428-a46e-64c9b0f4df27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:21.894578 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d93ff8-353d-4428-a46e-64c9b0f4df27 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:21.906367 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1287/2573] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:52:21.910686 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d4926-05b2-4442-85f8-4fb3575ac2cb None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:21.920679 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d4926-05b2-4442-85f8-4fb3575ac2cb None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:21.925715 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63e69e-6fe8-4e71-8c14-959cd274ccdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:21.926361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63e69e-6fe8-4e71-8c14-959cd274ccdb admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:21.926682 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63e69e-6fe8-4e71-8c14-959cd274ccdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:21.927221 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63e69e-6fe8-4e71-8c14-959cd274ccdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:21.927923 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63e69e-6fe8-4e71-8c14-959cd274ccdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:21.929809 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63e69e-6fe8-4e71-8c14-959cd274ccdb admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:22.004882 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1287/2574] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:52:21 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:22.008917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d93ff8-353d-4428-a46e-64c9b0f4df27 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:22.019562 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d93ff8-353d-4428-a46e-64c9b0f4df27 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:22.066934 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:22.076451 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108f471-e193-44f1-90c0-4902b8db44c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:22.077115 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108f471-e193-44f1-90c0-4902b8db44c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:22.077396 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108f471-e193-44f1-90c0-4902b8db44c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:22.077668 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108f471-e193-44f1-90c0-4902b8db44c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:22.080719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4108f471-e193-44f1-90c0-4902b8db44c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:22.080719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4108f471-e193-44f1-90c0-4902b8db44c1 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:22.188123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1288/2575] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:52:22 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:22.192078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63e69e-6fe8-4e71-8c14-959cd274ccdb admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:22.202545 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63e69e-6fe8-4e71-8c14-959cd274ccdb admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:22.207825 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd1d4b-c776-447b-bc35-23c5969d8f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:22.208481 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd1d4b-c776-447b-bc35-23c5969d8f0e admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:22.208765 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd1d4b-c776-447b-bc35-23c5969d8f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:22.209046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd1d4b-c776-447b-bc35-23c5969d8f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:22.209766 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd1d4b-c776-447b-bc35-23c5969d8f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:22.211414 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd1d4b-c776-447b-bc35-23c5969d8f0e admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:22.221806 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1288/2576] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:22.226787 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108f471-e193-44f1-90c0-4902b8db44c1 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:22.238599 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108f471-e193-44f1-90c0-4902b8db44c1 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:22.245166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab97537-6b1d-452d-b977-80577eb8d7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:22.245835 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab97537-6b1d-452d-b977-80577eb8d7a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:22.246163 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab97537-6b1d-452d-b977-80577eb8d7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:22.246474 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab97537-6b1d-452d-b977-80577eb8d7a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:22.247232 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab97537-6b1d-452d-b977-80577eb8d7a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:22.249092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab97537-6b1d-452d-b977-80577eb8d7a0 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:22.260462 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1289/2577] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:22.264696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd1d4b-c776-447b-bc35-23c5969d8f0e admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:22.278511 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd1d4b-c776-447b-bc35-23c5969d8f0e admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:22.283353 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d1775-5634-480a-aa2a-803e52a86c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:22.284356 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d1775-5634-480a-aa2a-803e52a86c1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:22.284675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d1775-5634-480a-aa2a-803e52a86c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:22.285023 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d1775-5634-480a-aa2a-803e52a86c1a admin admin] PATH_INFO: `/v3/projects/a981ae734b1e4993bb440918b1a74fe7/users/48ac761d30de4fb49478a78ccf23ab92/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:22.317710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d1775-5634-480a-aa2a-803e52a86c1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a981ae734b1e4993bb440918b1a74fe7, user_id=48ac761d30de4fb49478a78ccf23ab92, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:22.317710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d1775-5634-480a-aa2a-803e52a86c1a admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:22.416690 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1289/2578] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:22 2022] PUT /identity/v3/projects/a981ae734b1e4993bb440918b1a74fe7/users/48ac761d30de4fb49478a78ccf23ab92/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:22.422056 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cb760-5bf1-4e24-b2e5-6c847d06c5ee None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:22.422372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cb760-5bf1-4e24-b2e5-6c847d06c5ee None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:22.422730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cb760-5bf1-4e24-b2e5-6c847d06c5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:22.440009 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-ea6cb760-5bf1-4e24-b2e5-6c847d06c5ee None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:22.441269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-ea6cb760-5bf1-4e24-b2e5-6c847d06c5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:22.500344 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1290/2579] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:22.509977 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d1775-5634-480a-aa2a-803e52a86c1a admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:22.511756 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-355d1775-5634-480a-aa2a-803e52a86c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:22.567197 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d1775-5634-480a-aa2a-803e52a86c1a admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:22.570876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f65a0-8b02-4674-8f33-b51c186f01b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:22.572053 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f65a0-8b02-4674-8f33-b51c186f01b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:22.572339 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f65a0-8b02-4674-8f33-b51c186f01b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:22.572616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f65a0-8b02-4674-8f33-b51c186f01b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:22.581461 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f65a0-8b02-4674-8f33-b51c186f01b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:22.582954 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f65a0-8b02-4674-8f33-b51c186f01b4 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:22.592663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1290/2580] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:52:26.759925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6cb760-5bf1-4e24-b2e5-6c847d06c5ee None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:26.761606 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-ea6cb760-5bf1-4e24-b2e5-6c847d06c5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:26.773799 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6cb760-5bf1-4e24-b2e5-6c847d06c5ee None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:26.779322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36945c1-8f68-4f3b-a859-79aecdc370c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:26.780261 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36945c1-8f68-4f3b-a859-79aecdc370c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:26.781419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36945c1-8f68-4f3b-a859-79aecdc370c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:26.781419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36945c1-8f68-4f3b-a859-79aecdc370c7 admin admin] PATH_INFO: `/v3/users/46f6212ba6f84b70b5944686c8f31def` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:26.782307 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36945c1-8f68-4f3b-a859-79aecdc370c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46f6212ba6f84b70b5944686c8f31def)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:26.784680 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36945c1-8f68-4f3b-a859-79aecdc370c7 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:27.004637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c36945c1-8f68-4f3b-a859-79aecdc370c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46f6212ba6f84b70b5944686c8f31def'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:27.013974 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c36945c1-8f68-4f3b-a859-79aecdc370c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46f6212ba6f84b70b5944686c8f31def'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:27.015575 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c36945c1-8f68-4f3b-a859-79aecdc370c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46f6212ba6f84b70b5944686c8f31def'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:27.030471 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c36945c1-8f68-4f3b-a859-79aecdc370c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46f6212ba6f84b70b5944686c8f31def'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:27.089888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c36945c1-8f68-4f3b-a859-79aecdc370c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46f6212ba6f84b70b5944686c8f31def'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:27.098049 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1291/2581] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:52:26 2022] DELETE /identity/v3/users/46f6212ba6f84b70b5944686c8f31def => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:27.103171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a11c15-5fc8-44da-b699-d7332a5d0b86 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:27.104166 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a11c15-5fc8-44da-b699-d7332a5d0b86 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:27.104456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a11c15-5fc8-44da-b699-d7332a5d0b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:27.123432 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-b7a11c15-5fc8-44da-b699-d7332a5d0b86 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:27.124888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-b7a11c15-5fc8-44da-b699-d7332a5d0b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:27.209589 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1291/2582] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:27.211942 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36945c1-8f68-4f3b-a859-79aecdc370c7 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:27.213706 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-c36945c1-8f68-4f3b-a859-79aecdc370c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:27.271372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:27.292906 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36945c1-8f68-4f3b-a859-79aecdc370c7 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:27.295402 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2cad7e-dfa6-49eb-ae27-f8ad8c139c24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:27.296239 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cad7e-dfa6-49eb-ae27-f8ad8c139c24 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:27.296591 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cad7e-dfa6-49eb-ae27-f8ad8c139c24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:27.296913 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cad7e-dfa6-49eb-ae27-f8ad8c139c24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:27.308357 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2cad7e-dfa6-49eb-ae27-f8ad8c139c24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:27.309954 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2cad7e-dfa6-49eb-ae27-f8ad8c139c24 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:27.325083 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1292/2583] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:52:28.146137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a11c15-5fc8-44da-b699-d7332a5d0b86 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:28.148250 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-b7a11c15-5fc8-44da-b699-d7332a5d0b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:28.180061 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a11c15-5fc8-44da-b699-d7332a5d0b86 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:28.186083 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ee8c1-913a-48f7-9ae3-4d219d9d6948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:28.187065 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ee8c1-913a-48f7-9ae3-4d219d9d6948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:28.187391 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ee8c1-913a-48f7-9ae3-4d219d9d6948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:28.187809 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ee8c1-913a-48f7-9ae3-4d219d9d6948 admin admin] PATH_INFO: `/v3/projects/c6355ed0a51243d283a1092855634f7f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:28.189068 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ee8c1-913a-48f7-9ae3-4d219d9d6948 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6355ed0a51243d283a1092855634f7f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:28.191222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ee8c1-913a-48f7-9ae3-4d219d9d6948 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:28.204179 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:28.312857 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-766ee8c1-913a-48f7-9ae3-4d219d9d6948 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6355ed0a51243d283a1092855634f7f', 'request_id': 'req-766ee8c1-913a-48f7-9ae3-4d219d9d6948'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:28.393446 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-766ee8c1-913a-48f7-9ae3-4d219d9d6948 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6355ed0a51243d283a1092855634f7f', 'request_id': 'req-766ee8c1-913a-48f7-9ae3-4d219d9d6948'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:28.401824 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-766ee8c1-913a-48f7-9ae3-4d219d9d6948 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6355ed0a51243d283a1092855634f7f', 'request_id': 'req-766ee8c1-913a-48f7-9ae3-4d219d9d6948'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:28.416832 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-766ee8c1-913a-48f7-9ae3-4d219d9d6948 admin admin] The token cache is being invalidate because project c6355ed0a51243d283a1092855634f7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:52:28.416832 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-766ee8c1-913a-48f7-9ae3-4d219d9d6948 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:28.416832 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-766ee8c1-913a-48f7-9ae3-4d219d9d6948 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:28.417741 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1292/2584] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:52:28 2022] DELETE /identity/v3/projects/c6355ed0a51243d283a1092855634f7f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:28.422611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2cad7e-dfa6-49eb-ae27-f8ad8c139c24 service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:28.434038 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2cad7e-dfa6-49eb-ae27-f8ad8c139c24 service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:28.496616 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62ba16-4be1-468f-985b-ae4e79ab990e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:28.502410 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62ba16-4be1-468f-985b-ae4e79ab990e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:28.502410 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62ba16-4be1-468f-985b-ae4e79ab990e admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:28.502410 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62ba16-4be1-468f-985b-ae4e79ab990e admin admin] PATH_INFO: `/v3/users/9227a6272fd54efa889097ec795fa088` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:28.502410 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62ba16-4be1-468f-985b-ae4e79ab990e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9227a6272fd54efa889097ec795fa088)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:28.502410 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62ba16-4be1-468f-985b-ae4e79ab990e admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:28.725361 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-8c62ba16-4be1-468f-985b-ae4e79ab990e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9227a6272fd54efa889097ec795fa088'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:28.741858 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-8c62ba16-4be1-468f-985b-ae4e79ab990e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9227a6272fd54efa889097ec795fa088'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:28.749527 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-8c62ba16-4be1-468f-985b-ae4e79ab990e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9227a6272fd54efa889097ec795fa088'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:28.764174 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-8c62ba16-4be1-468f-985b-ae4e79ab990e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9227a6272fd54efa889097ec795fa088'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:28.822285 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-8c62ba16-4be1-468f-985b-ae4e79ab990e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9227a6272fd54efa889097ec795fa088'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:28.833035 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1293/2585] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:52:28 2022] DELETE /identity/v3/users/9227a6272fd54efa889097ec795fa088 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:29.415987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ee8c1-913a-48f7-9ae3-4d219d9d6948 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:29.415987 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-766ee8c1-913a-48f7-9ae3-4d219d9d6948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:29.440988 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ee8c1-913a-48f7-9ae3-4d219d9d6948 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:29.530442 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aca6d7-1fb3-466c-9e0b-58dd0c894908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:29.531923 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aca6d7-1fb3-466c-9e0b-58dd0c894908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:29.532414 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aca6d7-1fb3-466c-9e0b-58dd0c894908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:29.532826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aca6d7-1fb3-466c-9e0b-58dd0c894908 admin admin] PATH_INFO: `/v3/projects/3f2b66a8827048619123cfb55483d869` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:29.534291 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aca6d7-1fb3-466c-9e0b-58dd0c894908 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f2b66a8827048619123cfb55483d869)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:29.536512 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aca6d7-1fb3-466c-9e0b-58dd0c894908 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:29.686943 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-e4aca6d7-1fb3-466c-9e0b-58dd0c894908 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f2b66a8827048619123cfb55483d869', 'request_id': 'req-e4aca6d7-1fb3-466c-9e0b-58dd0c894908'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:29.735928 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-e4aca6d7-1fb3-466c-9e0b-58dd0c894908 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f2b66a8827048619123cfb55483d869', 'request_id': 'req-e4aca6d7-1fb3-466c-9e0b-58dd0c894908'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:29.747278 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-e4aca6d7-1fb3-466c-9e0b-58dd0c894908 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f2b66a8827048619123cfb55483d869', 'request_id': 'req-e4aca6d7-1fb3-466c-9e0b-58dd0c894908'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:29.764689 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-e4aca6d7-1fb3-466c-9e0b-58dd0c894908 admin admin] The token cache is being invalidate because project 3f2b66a8827048619123cfb55483d869 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:52:29.765101 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-e4aca6d7-1fb3-466c-9e0b-58dd0c894908 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:29.765863 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-e4aca6d7-1fb3-466c-9e0b-58dd0c894908 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:29.769292 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1293/2586] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:52:29 2022] DELETE /identity/v3/projects/3f2b66a8827048619123cfb55483d869 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:29.774452 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256521b6-e9da-498a-b2fe-971c0cbaa685 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:29.775484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256521b6-e9da-498a-b2fe-971c0cbaa685 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:29.776059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256521b6-e9da-498a-b2fe-971c0cbaa685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:29.802194 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-256521b6-e9da-498a-b2fe-971c0cbaa685 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:29.803493 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-256521b6-e9da-498a-b2fe-971c0cbaa685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:29.886730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1294/2587] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:29.895287 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54568b2-032c-4f26-9d4b-a174aeaf8c07 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:29.895658 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54568b2-032c-4f26-9d4b-a174aeaf8c07 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:29.896007 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54568b2-032c-4f26-9d4b-a174aeaf8c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:29.923716 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-a54568b2-032c-4f26-9d4b-a174aeaf8c07 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:29.925577 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-a54568b2-032c-4f26-9d4b-a174aeaf8c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:29.943011 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1294/2588] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:29.947027 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256521b6-e9da-498a-b2fe-971c0cbaa685 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:29.953991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:29.959375 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256521b6-e9da-498a-b2fe-971c0cbaa685 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:29.966656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617417b-bab2-4adc-9c06-30d3d2fc90c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:29.969369 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617417b-bab2-4adc-9c06-30d3d2fc90c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:29.969884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617417b-bab2-4adc-9c06-30d3d2fc90c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:29.970674 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617417b-bab2-4adc-9c06-30d3d2fc90c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:29.972570 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f617417b-bab2-4adc-9c06-30d3d2fc90c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:52:29.972971 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f617417b-bab2-4adc-9c06-30d3d2fc90c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:29.974926 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f617417b-bab2-4adc-9c06-30d3d2fc90c3 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:29.990768 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1295/2589] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:52:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:52:29.996522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54568b2-032c-4f26-9d4b-a174aeaf8c07 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:30.008984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54568b2-032c-4f26-9d4b-a174aeaf8c07 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:30.017188 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713a42b-9e26-4d1b-8e94-d80f3dea353a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:30.017988 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713a42b-9e26-4d1b-8e94-d80f3dea353a admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:30.018349 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713a42b-9e26-4d1b-8e94-d80f3dea353a admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:30.018722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713a42b-9e26-4d1b-8e94-d80f3dea353a admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:30.020214 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713a42b-9e26-4d1b-8e94-d80f3dea353a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:30.022919 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713a42b-9e26-4d1b-8e94-d80f3dea353a admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:30.099899 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1295/2590] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:52:29 2022] POST /identity/v3/projects => generated 401 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:30.104658 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617417b-bab2-4adc-9c06-30d3d2fc90c3 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:30.115395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617417b-bab2-4adc-9c06-30d3d2fc90c3 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:30.213884 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfaff2-a498-4918-a1f1-16f11b36c28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:30.214650 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfaff2-a498-4918-a1f1-16f11b36c28a admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:30.214941 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfaff2-a498-4918-a1f1-16f11b36c28a admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:30.215222 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfaff2-a498-4918-a1f1-16f11b36c28a admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:30.215938 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cfaff2-a498-4918-a1f1-16f11b36c28a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:30.218119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cfaff2-a498-4918-a1f1-16f11b36c28a admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:30.336207 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1296/2591] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:52:30 2022] POST /identity/v3/users => generated 301 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:30.340963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713a42b-9e26-4d1b-8e94-d80f3dea353a admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:30.346841 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:30.352137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713a42b-9e26-4d1b-8e94-d80f3dea353a admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:30.357365 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb813e-32a6-4e0d-9cff-22cdbbe45fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:30.358024 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb813e-32a6-4e0d-9cff-22cdbbe45fe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:30.358307 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb813e-32a6-4e0d-9cff-22cdbbe45fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:30.358630 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb813e-32a6-4e0d-9cff-22cdbbe45fe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:30.359648 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb813e-32a6-4e0d-9cff-22cdbbe45fe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:30.361176 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb813e-32a6-4e0d-9cff-22cdbbe45fe3 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:30.373947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1296/2592] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:30.377998 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfaff2-a498-4918-a1f1-16f11b36c28a admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:30.389100 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfaff2-a498-4918-a1f1-16f11b36c28a admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:30.393831 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50a280-14e0-47ef-abe2-846f75e12ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:30.394593 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50a280-14e0-47ef-abe2-846f75e12ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:30.394941 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50a280-14e0-47ef-abe2-846f75e12ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:30.395305 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50a280-14e0-47ef-abe2-846f75e12ea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:30.396109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50a280-14e0-47ef-abe2-846f75e12ea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:30.398949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50a280-14e0-47ef-abe2-846f75e12ea0 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:30.414674 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb813e-32a6-4e0d-9cff-22cdbbe45fe3 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:30.416443 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1297/2593] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:30.427400 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb813e-32a6-4e0d-9cff-22cdbbe45fe3 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:30.433488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ad729-22cd-425d-88e8-266c84b94686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:30.434504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ad729-22cd-425d-88e8-266c84b94686 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:30.434848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ad729-22cd-425d-88e8-266c84b94686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:30.435183 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ad729-22cd-425d-88e8-266c84b94686 admin admin] PATH_INFO: `/v3/projects/69946c874e594890a4e4804a047b7578/users/4bdf4bde9dc64a02ab7f2f43becb0129/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:30.451959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ad729-22cd-425d-88e8-266c84b94686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69946c874e594890a4e4804a047b7578, user_id=4bdf4bde9dc64a02ab7f2f43becb0129, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:30.454957 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ad729-22cd-425d-88e8-266c84b94686 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:30.551348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1297/2594] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:30 2022] PUT /identity/v3/projects/69946c874e594890a4e4804a047b7578/users/4bdf4bde9dc64a02ab7f2f43becb0129/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:52:30.562707 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7829efa-d15a-4a13-8811-22a2dd62e722 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:30.563064 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7829efa-d15a-4a13-8811-22a2dd62e722 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:30.563411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7829efa-d15a-4a13-8811-22a2dd62e722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:30.604918 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-f7829efa-d15a-4a13-8811-22a2dd62e722 None None] MFA Rules not processed for user `4bdf4bde9dc64a02ab7f2f43becb0129`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:30.606149 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f7829efa-d15a-4a13-8811-22a2dd62e722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:30.683938 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1298/2595] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:30.688438 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ad729-22cd-425d-88e8-266c84b94686 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:30.791055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ad729-22cd-425d-88e8-266c84b94686 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:30.798137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f0c9b-2383-4ec8-9018-98871cb15760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:30.798911 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f0c9b-2383-4ec8-9018-98871cb15760 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:30.799316 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f0c9b-2383-4ec8-9018-98871cb15760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:30.799457 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f0c9b-2383-4ec8-9018-98871cb15760 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:30.800230 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f0c9b-2383-4ec8-9018-98871cb15760 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:30.802425 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f0c9b-2383-4ec8-9018-98871cb15760 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:30.813442 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc996a-92b5-4e8b-a378-72daea5e7d03 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:30.813920 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc996a-92b5-4e8b-a378-72daea5e7d03 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:30.814127 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc996a-92b5-4e8b-a378-72daea5e7d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:30.845866 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-eddc996a-92b5-4e8b-a378-72daea5e7d03 None None] MFA Rules not processed for user `48ac761d30de4fb49478a78ccf23ab92`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:30.847899 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-eddc996a-92b5-4e8b-a378-72daea5e7d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:30.876501 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1298/2596] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:52:30 2022] POST /identity/v3/projects => generated 399 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:30.880448 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f0c9b-2383-4ec8-9018-98871cb15760 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:30.924100 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:30.959175 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1299/2597] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:30.979494 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc996a-92b5-4e8b-a378-72daea5e7d03 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:30.979494 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-eddc996a-92b5-4e8b-a378-72daea5e7d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:30.979659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f0c9b-2383-4ec8-9018-98871cb15760 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:30.979659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15d018-6d4e-45ec-911b-13fae177cb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:30.979659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15d018-6d4e-45ec-911b-13fae177cb55 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:30.979659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15d018-6d4e-45ec-911b-13fae177cb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:30.980078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15d018-6d4e-45ec-911b-13fae177cb55 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:30.980078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15d018-6d4e-45ec-911b-13fae177cb55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:30.980078 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15d018-6d4e-45ec-911b-13fae177cb55 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:31.052604 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc996a-92b5-4e8b-a378-72daea5e7d03 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:31.084900 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1299/2598] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:52:30 2022] POST /identity/v3/users => generated 306 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:52:31.088898 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15d018-6d4e-45ec-911b-13fae177cb55 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:31.099073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15d018-6d4e-45ec-911b-13fae177cb55 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:31.104761 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74e5f0-b5f9-4910-bcdc-5af08f08913e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:31.105541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74e5f0-b5f9-4910-bcdc-5af08f08913e admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:31.105844 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74e5f0-b5f9-4910-bcdc-5af08f08913e admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:31.106419 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74e5f0-b5f9-4910-bcdc-5af08f08913e admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:31.107171 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74e5f0-b5f9-4910-bcdc-5af08f08913e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:31.110011 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74e5f0-b5f9-4910-bcdc-5af08f08913e admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:31.120254 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ac257-a8e8-4d0b-b017-0c7eede73fd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:31.120901 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ac257-a8e8-4d0b-b017-0c7eede73fd6 service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:31.121186 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ac257-a8e8-4d0b-b017-0c7eede73fd6 service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:31.124436 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1300/2599] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:31.124571 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ac257-a8e8-4d0b-b017-0c7eede73fd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:31.126214 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74e5f0-b5f9-4910-bcdc-5af08f08913e admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:31.137042 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74e5f0-b5f9-4910-bcdc-5af08f08913e admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:31.142276 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fdf9cd-b5c7-4afb-8d6e-045aa41610c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:31.143417 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fdf9cd-b5c7-4afb-8d6e-045aa41610c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:31.143904 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fdf9cd-b5c7-4afb-8d6e-045aa41610c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:31.144251 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fdf9cd-b5c7-4afb-8d6e-045aa41610c1 admin admin] PATH_INFO: `/v3/projects/353448f83ad84d5c95980ddd5f97eccc/users/60b44c3dae5b403daa468ca9325abd29/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:31.161501 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fdf9cd-b5c7-4afb-8d6e-045aa41610c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=353448f83ad84d5c95980ddd5f97eccc, user_id=60b44c3dae5b403daa468ca9325abd29, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:31.164468 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fdf9cd-b5c7-4afb-8d6e-045aa41610c1 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:31.182357 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ac257-a8e8-4d0b-b017-0c7eede73fd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:31.183999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ac257-a8e8-4d0b-b017-0c7eede73fd6 service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:31.193028 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1300/2600] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:52:31.262902 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1301/2601] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:31 2022] PUT /identity/v3/projects/353448f83ad84d5c95980ddd5f97eccc/users/60b44c3dae5b403daa468ca9325abd29/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:31.267142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ac257-a8e8-4d0b-b017-0c7eede73fd6 service nova] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:31.279149 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ac257-a8e8-4d0b-b017-0c7eede73fd6 service nova] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:31.341104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37398366-6922-4f44-b49a-129f13c82cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:31.341889 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37398366-6922-4f44-b49a-129f13c82cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:31.342276 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37398366-6922-4f44-b49a-129f13c82cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:31.342706 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37398366-6922-4f44-b49a-129f13c82cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:31.343962 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37398366-6922-4f44-b49a-129f13c82cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:31.345372 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37398366-6922-4f44-b49a-129f13c82cd5 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:31.358946 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1301/2602] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:52:31.373222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fdf9cd-b5c7-4afb-8d6e-045aa41610c1 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:31.384086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fdf9cd-b5c7-4afb-8d6e-045aa41610c1 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:31.388945 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beee848-dc44-45d5-afbb-6b641d49fc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:31.389978 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beee848-dc44-45d5-afbb-6b641d49fc8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:31.390367 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beee848-dc44-45d5-afbb-6b641d49fc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:31.390775 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beee848-dc44-45d5-afbb-6b641d49fc8d admin admin] PATH_INFO: `/v3/projects/353448f83ad84d5c95980ddd5f97eccc/users/60b44c3dae5b403daa468ca9325abd29/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:31.392970 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beee848-dc44-45d5-afbb-6b641d49fc8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=353448f83ad84d5c95980ddd5f97eccc, user_id=60b44c3dae5b403daa468ca9325abd29, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:31.395787 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beee848-dc44-45d5-afbb-6b641d49fc8d admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:31.398405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:31.408372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1302/2603] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:31 2022] PUT /identity/v3/projects/353448f83ad84d5c95980ddd5f97eccc/users/60b44c3dae5b403daa468ca9325abd29/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:31.420114 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b6db9-21fc-4e23-997b-5cc40a1abbce None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:31.420671 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b6db9-21fc-4e23-997b-5cc40a1abbce None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:31.420954 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b6db9-21fc-4e23-997b-5cc40a1abbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:31.449094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-6c8b6db9-21fc-4e23-997b-5cc40a1abbce None None] MFA Rules not processed for user `60b44c3dae5b403daa468ca9325abd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:31.450180 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-6c8b6db9-21fc-4e23-997b-5cc40a1abbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:31.533841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1302/2604] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:31 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:31.546034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beee848-dc44-45d5-afbb-6b641d49fc8d admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:31.547890 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-8beee848-dc44-45d5-afbb-6b641d49fc8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:31.627454 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beee848-dc44-45d5-afbb-6b641d49fc8d admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:31.631450 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01babe3e-e0ad-472b-bf5e-3d3416397337 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:31.632265 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01babe3e-e0ad-472b-bf5e-3d3416397337 service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:31.637348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01babe3e-e0ad-472b-bf5e-3d3416397337 service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:31.637348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01babe3e-e0ad-472b-bf5e-3d3416397337 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:31.645624 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01babe3e-e0ad-472b-bf5e-3d3416397337 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:31.647134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01babe3e-e0ad-472b-bf5e-3d3416397337 service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:31.660224 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1303/2605] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:31 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:52:31.685672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:31.899364 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b6db9-21fc-4e23-997b-5cc40a1abbce None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:31.913825 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b6db9-21fc-4e23-997b-5cc40a1abbce None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:31.920907 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe690ba-b610-49dd-bdc7-7cf88ef3113d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:31.921574 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe690ba-b610-49dd-bdc7-7cf88ef3113d service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:31.921855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe690ba-b610-49dd-bdc7-7cf88ef3113d service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:31.922672 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe690ba-b610-49dd-bdc7-7cf88ef3113d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:31.996542 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe690ba-b610-49dd-bdc7-7cf88ef3113d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:31.997982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe690ba-b610-49dd-bdc7-7cf88ef3113d service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:32.009313 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1303/2606] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:31 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:52:34.085163 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0990f-9419-43de-8efd-5a10548f9dcc None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:34.085660 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0990f-9419-43de-8efd-5a10548f9dcc None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:34.085759 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0990f-9419-43de-8efd-5a10548f9dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:34.104888 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-5eb0990f-9419-43de-8efd-5a10548f9dcc None None] MFA Rules not processed for user `48ac761d30de4fb49478a78ccf23ab92`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:34.106210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-5eb0990f-9419-43de-8efd-5a10548f9dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:34.163231 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1304/2607] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:37.177726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe690ba-b610-49dd-bdc7-7cf88ef3113d service cinder] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:37.223984 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:37.240933 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe690ba-b610-49dd-bdc7-7cf88ef3113d service cinder] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:37.246427 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f96c1-0158-4b4c-a99a-bd9f1c18a928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:37.247275 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f96c1-0158-4b4c-a99a-bd9f1c18a928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:37.247555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f96c1-0158-4b4c-a99a-bd9f1c18a928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:37.247827 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f96c1-0158-4b4c-a99a-bd9f1c18a928 admin admin] PATH_INFO: `/v3/users/4bdf4bde9dc64a02ab7f2f43becb0129` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:37.249077 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f96c1-0158-4b4c-a99a-bd9f1c18a928 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bdf4bde9dc64a02ab7f2f43becb0129)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:37.251087 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f96c1-0158-4b4c-a99a-bd9f1c18a928 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:37.447750 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f19f96c1-0158-4b4c-a99a-bd9f1c18a928 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bdf4bde9dc64a02ab7f2f43becb0129'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:37.454930 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f19f96c1-0158-4b4c-a99a-bd9f1c18a928 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bdf4bde9dc64a02ab7f2f43becb0129'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:37.455640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f19f96c1-0158-4b4c-a99a-bd9f1c18a928 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bdf4bde9dc64a02ab7f2f43becb0129'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:37.469752 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f19f96c1-0158-4b4c-a99a-bd9f1c18a928 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bdf4bde9dc64a02ab7f2f43becb0129'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:37.518812 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-f19f96c1-0158-4b4c-a99a-bd9f1c18a928 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bdf4bde9dc64a02ab7f2f43becb0129'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:37.526461 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1304/2608] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:52:37 2022] DELETE /identity/v3/users/4bdf4bde9dc64a02ab7f2f43becb0129 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:37.530675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574da74-ba40-437a-82e4-aae503e348be None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:37.531168 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574da74-ba40-437a-82e4-aae503e348be None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:37.531262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574da74-ba40-437a-82e4-aae503e348be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:37.550160 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-c574da74-ba40-437a-82e4-aae503e348be None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:37.551418 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c574da74-ba40-437a-82e4-aae503e348be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:37.616024 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1305/2609] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:37.625989 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f96c1-0158-4b4c-a99a-bd9f1c18a928 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:37.627466 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f19f96c1-0158-4b4c-a99a-bd9f1c18a928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:37.707490 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f96c1-0158-4b4c-a99a-bd9f1c18a928 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:37.712554 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95c164-daae-4b7f-b926-5ae304295182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:37.713308 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95c164-daae-4b7f-b926-5ae304295182 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:37.713601 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95c164-daae-4b7f-b926-5ae304295182 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:37.715902 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95c164-daae-4b7f-b926-5ae304295182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:37.726635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95c164-daae-4b7f-b926-5ae304295182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:37.728967 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95c164-daae-4b7f-b926-5ae304295182 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:37.740905 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1305/2610] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:52:38.592325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574da74-ba40-437a-82e4-aae503e348be None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:38.593970 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c574da74-ba40-437a-82e4-aae503e348be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:38.610480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574da74-ba40-437a-82e4-aae503e348be None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:38.617847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070d179-61c6-44f1-9b42-f53bbc2fc41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:38.619047 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070d179-61c6-44f1-9b42-f53bbc2fc41d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:38.619492 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070d179-61c6-44f1-9b42-f53bbc2fc41d admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:38.619988 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070d179-61c6-44f1-9b42-f53bbc2fc41d admin admin] PATH_INFO: `/v3/projects/69946c874e594890a4e4804a047b7578` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:38.621426 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f070d179-61c6-44f1-9b42-f53bbc2fc41d admin admin] RBAC: Authorizing `identity:delete_project(project_id=69946c874e594890a4e4804a047b7578)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:38.623613 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f070d179-61c6-44f1-9b42-f53bbc2fc41d admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:38.722541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:INSERT=1,keystone:SELECT=96,keystone:DELETE=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:38.740920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f070d179-61c6-44f1-9b42-f53bbc2fc41d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69946c874e594890a4e4804a047b7578', 'request_id': 'req-f070d179-61c6-44f1-9b42-f53bbc2fc41d'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:38.810880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f070d179-61c6-44f1-9b42-f53bbc2fc41d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69946c874e594890a4e4804a047b7578', 'request_id': 'req-f070d179-61c6-44f1-9b42-f53bbc2fc41d'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:38.817827 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f070d179-61c6-44f1-9b42-f53bbc2fc41d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69946c874e594890a4e4804a047b7578', 'request_id': 'req-f070d179-61c6-44f1-9b42-f53bbc2fc41d'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:38.828696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f070d179-61c6-44f1-9b42-f53bbc2fc41d admin admin] The token cache is being invalidate because project 69946c874e594890a4e4804a047b7578 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:52:38.829042 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f070d179-61c6-44f1-9b42-f53bbc2fc41d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:38.829724 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f070d179-61c6-44f1-9b42-f53bbc2fc41d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:38.831425 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1306/2611] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:52:38 2022] DELETE /identity/v3/projects/69946c874e594890a4e4804a047b7578 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:38.835576 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95c164-daae-4b7f-b926-5ae304295182 service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:38.845794 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95c164-daae-4b7f-b926-5ae304295182 service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:38.907777 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ab47f-5b5a-495d-8d8a-ed72ad04499d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:38.908637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ab47f-5b5a-495d-8d8a-ed72ad04499d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:38.908911 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ab47f-5b5a-495d-8d8a-ed72ad04499d admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:38.909178 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ab47f-5b5a-495d-8d8a-ed72ad04499d admin admin] PATH_INFO: `/v3/users/60b44c3dae5b403daa468ca9325abd29` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:38.910350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ab47f-5b5a-495d-8d8a-ed72ad04499d admin admin] RBAC: Authorizing `identity:delete_user(user_id=60b44c3dae5b403daa468ca9325abd29)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:38.912382 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ab47f-5b5a-495d-8d8a-ed72ad04499d admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:38.918322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:39.116338 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-3c3ab47f-5b5a-495d-8d8a-ed72ad04499d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60b44c3dae5b403daa468ca9325abd29'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:39.126771 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-3c3ab47f-5b5a-495d-8d8a-ed72ad04499d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60b44c3dae5b403daa468ca9325abd29'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:39.127607 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-3c3ab47f-5b5a-495d-8d8a-ed72ad04499d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60b44c3dae5b403daa468ca9325abd29'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:39.143997 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-3c3ab47f-5b5a-495d-8d8a-ed72ad04499d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60b44c3dae5b403daa468ca9325abd29'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:39.210209 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-3c3ab47f-5b5a-495d-8d8a-ed72ad04499d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60b44c3dae5b403daa468ca9325abd29'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:39.216991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1306/2612] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:52:38 2022] DELETE /identity/v3/users/60b44c3dae5b403daa468ca9325abd29 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:39.774764 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070d179-61c6-44f1-9b42-f53bbc2fc41d admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:39.776481 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-f070d179-61c6-44f1-9b42-f53bbc2fc41d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:39.794268 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070d179-61c6-44f1-9b42-f53bbc2fc41d admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:39.876011 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700451b2-b193-4a3c-8cdc-061791e6718c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:39.877022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700451b2-b193-4a3c-8cdc-061791e6718c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:39.877368 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700451b2-b193-4a3c-8cdc-061791e6718c admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:39.878464 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700451b2-b193-4a3c-8cdc-061791e6718c admin admin] PATH_INFO: `/v3/projects/353448f83ad84d5c95980ddd5f97eccc` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:39.879094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700451b2-b193-4a3c-8cdc-061791e6718c admin admin] RBAC: Authorizing `identity:delete_project(project_id=353448f83ad84d5c95980ddd5f97eccc)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:39.883348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700451b2-b193-4a3c-8cdc-061791e6718c admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:40.007626 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-700451b2-b193-4a3c-8cdc-061791e6718c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '353448f83ad84d5c95980ddd5f97eccc', 'request_id': 'req-700451b2-b193-4a3c-8cdc-061791e6718c'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:40.107252 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-700451b2-b193-4a3c-8cdc-061791e6718c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '353448f83ad84d5c95980ddd5f97eccc', 'request_id': 'req-700451b2-b193-4a3c-8cdc-061791e6718c'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:40.114295 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-700451b2-b193-4a3c-8cdc-061791e6718c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '353448f83ad84d5c95980ddd5f97eccc', 'request_id': 'req-700451b2-b193-4a3c-8cdc-061791e6718c'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:40.128134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-700451b2-b193-4a3c-8cdc-061791e6718c admin admin] The token cache is being invalidate because project 353448f83ad84d5c95980ddd5f97eccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:52:40.131768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-700451b2-b193-4a3c-8cdc-061791e6718c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:40.131768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-700451b2-b193-4a3c-8cdc-061791e6718c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:40.131768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1307/2613] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:52:39 2022] DELETE /identity/v3/projects/353448f83ad84d5c95980ddd5f97eccc => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:40.135571 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f16390-0006-46a6-ae19-672882cf4a16 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:40.136394 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f16390-0006-46a6-ae19-672882cf4a16 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:40.136678 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f16390-0006-46a6-ae19-672882cf4a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:40.155088 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-f9f16390-0006-46a6-ae19-672882cf4a16 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:40.156408 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f9f16390-0006-46a6-ae19-672882cf4a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:40.222137 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1307/2614] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:40.230766 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce2563-dcaf-461e-bc0e-0eacec484d4a None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:40.231051 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce2563-dcaf-461e-bc0e-0eacec484d4a None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:40.231328 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce2563-dcaf-461e-bc0e-0eacec484d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:40.251788 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-24ce2563-dcaf-461e-bc0e-0eacec484d4a None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:40.253657 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-24ce2563-dcaf-461e-bc0e-0eacec484d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:40.261755 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1308/2615] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:40.265769 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f16390-0006-46a6-ae19-672882cf4a16 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:40.277825 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f16390-0006-46a6-ae19-672882cf4a16 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:40.282797 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c91ba1-98f1-4cd2-a9d2-f8f11d2b37be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:40.291398 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c91ba1-98f1-4cd2-a9d2-f8f11d2b37be admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:40.291398 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c91ba1-98f1-4cd2-a9d2-f8f11d2b37be admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:40.291398 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c91ba1-98f1-4cd2-a9d2-f8f11d2b37be admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:40.291398 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c91ba1-98f1-4cd2-a9d2-f8f11d2b37be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:52:40.291398 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c91ba1-98f1-4cd2-a9d2-f8f11d2b37be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:40.291398 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c91ba1-98f1-4cd2-a9d2-f8f11d2b37be admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:40.300283 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1308/2616] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:52:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:52:40.304558 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce2563-dcaf-461e-bc0e-0eacec484d4a None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:40.313938 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce2563-dcaf-461e-bc0e-0eacec484d4a None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:40.319924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbccc54-00e9-42b5-bfbb-fd6ad633e5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:40.319924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbccc54-00e9-42b5-bfbb-fd6ad633e5dd admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:40.319924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbccc54-00e9-42b5-bfbb-fd6ad633e5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:40.319924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbccc54-00e9-42b5-bfbb-fd6ad633e5dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:40.320481 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbccc54-00e9-42b5-bfbb-fd6ad633e5dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:40.322802 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbccc54-00e9-42b5-bfbb-fd6ad633e5dd admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:40.425956 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1309/2617] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:52:40 2022] POST /identity/v3/projects => generated 389 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:40.429205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c91ba1-98f1-4cd2-a9d2-f8f11d2b37be admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:40.442376 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c91ba1-98f1-4cd2-a9d2-f8f11d2b37be admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:40.509686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2893e-65de-4a4c-9e78-996af43932f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:40.510732 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2893e-65de-4a4c-9e78-996af43932f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:40.511022 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2893e-65de-4a4c-9e78-996af43932f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:40.511664 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2893e-65de-4a4c-9e78-996af43932f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:40.512342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba2893e-65de-4a4c-9e78-996af43932f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:40.515102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba2893e-65de-4a4c-9e78-996af43932f5 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:40.619197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1309/2618] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:52:40 2022] POST /identity/v3/users => generated 295 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:40.621055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbccc54-00e9-42b5-bfbb-fd6ad633e5dd admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:40.631059 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbccc54-00e9-42b5-bfbb-fd6ad633e5dd admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:40.636471 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1e22f-68ac-4d4b-a7b4-4f1c1ec7850a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:40.637125 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1e22f-68ac-4d4b-a7b4-4f1c1ec7850a admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:40.637406 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1e22f-68ac-4d4b-a7b4-4f1c1ec7850a admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:40.637678 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1e22f-68ac-4d4b-a7b4-4f1c1ec7850a admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:40.638472 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc1e22f-68ac-4d4b-a7b4-4f1c1ec7850a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:40.640176 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc1e22f-68ac-4d4b-a7b4-4f1c1ec7850a admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:40.652230 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1310/2619] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:40.656155 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2893e-65de-4a4c-9e78-996af43932f5 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:40.661278 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:40.666551 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2893e-65de-4a4c-9e78-996af43932f5 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:40.671344 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be7c266-770a-4109-8f8e-0bcee4cc8b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:40.672060 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7c266-770a-4109-8f8e-0bcee4cc8b66 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:40.673130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7c266-770a-4109-8f8e-0bcee4cc8b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:40.673130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7c266-770a-4109-8f8e-0bcee4cc8b66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:40.673442 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be7c266-770a-4109-8f8e-0bcee4cc8b66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:40.675095 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be7c266-770a-4109-8f8e-0bcee4cc8b66 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:40.687732 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1310/2620] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:40.691261 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1e22f-68ac-4d4b-a7b4-4f1c1ec7850a admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:40.701768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1e22f-68ac-4d4b-a7b4-4f1c1ec7850a admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:40.706468 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b144d-4051-4cc5-a9d9-f6fb9cabc4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:40.707349 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b144d-4051-4cc5-a9d9-f6fb9cabc4fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:40.707637 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b144d-4051-4cc5-a9d9-f6fb9cabc4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:40.707910 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b144d-4051-4cc5-a9d9-f6fb9cabc4fe admin admin] PATH_INFO: `/v3/projects/124cd6c5f6df40c5b20167f6a49c917d/users/5de7f3a65f574e9ba8dd0e0d9e83a5f9/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:40.721253 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:40.724266 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84b144d-4051-4cc5-a9d9-f6fb9cabc4fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=124cd6c5f6df40c5b20167f6a49c917d, user_id=5de7f3a65f574e9ba8dd0e0d9e83a5f9, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:40.727133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84b144d-4051-4cc5-a9d9-f6fb9cabc4fe admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:40.786506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1311/2621] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:40 2022] PUT /identity/v3/projects/124cd6c5f6df40c5b20167f6a49c917d/users/5de7f3a65f574e9ba8dd0e0d9e83a5f9/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:40.796963 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1168795-855e-4ffb-84cf-c3852d2290a2 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:40.797245 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1168795-855e-4ffb-84cf-c3852d2290a2 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:40.797518 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1168795-855e-4ffb-84cf-c3852d2290a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:40.823614 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-d1168795-855e-4ffb-84cf-c3852d2290a2 None None] MFA Rules not processed for user `5de7f3a65f574e9ba8dd0e0d9e83a5f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:40.824807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-d1168795-855e-4ffb-84cf-c3852d2290a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:40.894688 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1311/2622] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:40.899139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b144d-4051-4cc5-a9d9-f6fb9cabc4fe admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:40.964700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b144d-4051-4cc5-a9d9-f6fb9cabc4fe admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:40.969372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c3223-a230-4be0-bcba-fe6918612927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:40.970048 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c3223-a230-4be0-bcba-fe6918612927 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:40.970355 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c3223-a230-4be0-bcba-fe6918612927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:40.970657 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c3223-a230-4be0-bcba-fe6918612927 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:40.971357 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2c3223-a230-4be0-bcba-fe6918612927 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:40.973322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2c3223-a230-4be0-bcba-fe6918612927 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:41.063572 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1312/2623] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:52:40 2022] POST /identity/v3/projects => generated 389 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:52:41.067315 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1168795-855e-4ffb-84cf-c3852d2290a2 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:41.080807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1168795-855e-4ffb-84cf-c3852d2290a2 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:41.143743 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d1324-8afc-47f5-bb10-890fab178eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:41.144410 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d1324-8afc-47f5-bb10-890fab178eee admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:41.144697 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d1324-8afc-47f5-bb10-890fab178eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:41.144976 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d1324-8afc-47f5-bb10-890fab178eee admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:41.145659 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d1324-8afc-47f5-bb10-890fab178eee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:41.147613 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d1324-8afc-47f5-bb10-890fab178eee admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:41.221060 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1312/2624] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:52:41 2022] POST /identity/v3/users => generated 301 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:41.225261 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c3223-a230-4be0-bcba-fe6918612927 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:41.237500 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c3223-a230-4be0-bcba-fe6918612927 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:41.242419 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13039dd-9e00-4f4d-a336-4a345b8abfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:41.243406 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13039dd-9e00-4f4d-a336-4a345b8abfa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:41.243706 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13039dd-9e00-4f4d-a336-4a345b8abfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:41.244057 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13039dd-9e00-4f4d-a336-4a345b8abfa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:41.244813 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13039dd-9e00-4f4d-a336-4a345b8abfa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:41.246423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13039dd-9e00-4f4d-a336-4a345b8abfa6 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:41.257534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1313/2625] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:41.266183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d1324-8afc-47f5-bb10-890fab178eee admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:41.277422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d1324-8afc-47f5-bb10-890fab178eee admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:41.282242 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584b100-9c19-4b1f-8c3d-e3672342e4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:41.283251 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584b100-9c19-4b1f-8c3d-e3672342e4ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:41.283533 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584b100-9c19-4b1f-8c3d-e3672342e4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:41.283807 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584b100-9c19-4b1f-8c3d-e3672342e4ae admin admin] PATH_INFO: `/v3/projects/9ffe62941e94480cbcf2f90ef1318042/users/0c2ddb049ca34f5cb9e852e53ea776e0/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:41.300040 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7584b100-9c19-4b1f-8c3d-e3672342e4ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ffe62941e94480cbcf2f90ef1318042, user_id=0c2ddb049ca34f5cb9e852e53ea776e0, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:41.302738 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7584b100-9c19-4b1f-8c3d-e3672342e4ae admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:41.389117 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1313/2626] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:41 2022] PUT /identity/v3/projects/9ffe62941e94480cbcf2f90ef1318042/users/0c2ddb049ca34f5cb9e852e53ea776e0/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:41.393295 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13039dd-9e00-4f4d-a336-4a345b8abfa6 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:41.404350 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13039dd-9e00-4f4d-a336-4a345b8abfa6 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:41.460635 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b7eab3-a167-4955-8660-8609cd669267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:41.461292 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7eab3-a167-4955-8660-8609cd669267 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:41.461587 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7eab3-a167-4955-8660-8609cd669267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:41.461855 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7eab3-a167-4955-8660-8609cd669267 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:41.463283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b7eab3-a167-4955-8660-8609cd669267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:41.464884 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b7eab3-a167-4955-8660-8609cd669267 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:41.475039 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1314/2627] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:41.480001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584b100-9c19-4b1f-8c3d-e3672342e4ae admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:41.491534 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584b100-9c19-4b1f-8c3d-e3672342e4ae admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:41.501430 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745f456-1e61-4dae-a210-fa64f8cee1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:41.501430 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745f456-1e61-4dae-a210-fa64f8cee1a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:41.501430 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745f456-1e61-4dae-a210-fa64f8cee1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:41.501430 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745f456-1e61-4dae-a210-fa64f8cee1a9 admin admin] PATH_INFO: `/v3/projects/9ffe62941e94480cbcf2f90ef1318042/users/0c2ddb049ca34f5cb9e852e53ea776e0/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:41.501838 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745f456-1e61-4dae-a210-fa64f8cee1a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ffe62941e94480cbcf2f90ef1318042, user_id=0c2ddb049ca34f5cb9e852e53ea776e0, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:41.503121 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745f456-1e61-4dae-a210-fa64f8cee1a9 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:41.514166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1314/2628] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:41 2022] PUT /identity/v3/projects/9ffe62941e94480cbcf2f90ef1318042/users/0c2ddb049ca34f5cb9e852e53ea776e0/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:41.526001 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdb342-2e0a-4fd6-b0f6-e7f053dad3af None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:41.526245 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdb342-2e0a-4fd6-b0f6-e7f053dad3af None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:41.526690 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdb342-2e0a-4fd6-b0f6-e7f053dad3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:41.553302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-ddfdb342-2e0a-4fd6-b0f6-e7f053dad3af None None] MFA Rules not processed for user `0c2ddb049ca34f5cb9e852e53ea776e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:41.554537 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ddfdb342-2e0a-4fd6-b0f6-e7f053dad3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:41.600864 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:41.620874 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1315/2629] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:52:41.631622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745f456-1e61-4dae-a210-fa64f8cee1a9 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:41.633354 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-9745f456-1e61-4dae-a210-fa64f8cee1a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:41.668827 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:41.691614 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745f456-1e61-4dae-a210-fa64f8cee1a9 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:41.695411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ac8ad-bcdb-4aac-85ec-cc031e71dee8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:41.696051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ac8ad-bcdb-4aac-85ec-cc031e71dee8 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:41.696333 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ac8ad-bcdb-4aac-85ec-cc031e71dee8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:41.696606 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ac8ad-bcdb-4aac-85ec-cc031e71dee8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:41.706604 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ac8ad-bcdb-4aac-85ec-cc031e71dee8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:41.708033 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ac8ad-bcdb-4aac-85ec-cc031e71dee8 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:41.719028 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1315/2630] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:52:41.854047 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdb342-2e0a-4fd6-b0f6-e7f053dad3af None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:41.928612 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdb342-2e0a-4fd6-b0f6-e7f053dad3af None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:41.933428 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c128d1-5274-4fb9-b28e-55843ae296ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:41.934623 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c128d1-5274-4fb9-b28e-55843ae296ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:41.934939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c128d1-5274-4fb9-b28e-55843ae296ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:41.935296 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c128d1-5274-4fb9-b28e-55843ae296ea admin admin] PATH_INFO: `/v3/users/5de7f3a65f574e9ba8dd0e0d9e83a5f9` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:41.936596 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c128d1-5274-4fb9-b28e-55843ae296ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=5de7f3a65f574e9ba8dd0e0d9e83a5f9)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:41.938576 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c128d1-5274-4fb9-b28e-55843ae296ea admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:42.166231 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f8c128d1-5274-4fb9-b28e-55843ae296ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5de7f3a65f574e9ba8dd0e0d9e83a5f9'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:42.173378 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f8c128d1-5274-4fb9-b28e-55843ae296ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5de7f3a65f574e9ba8dd0e0d9e83a5f9'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:42.173982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f8c128d1-5274-4fb9-b28e-55843ae296ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5de7f3a65f574e9ba8dd0e0d9e83a5f9'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:42.174668 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f8c128d1-5274-4fb9-b28e-55843ae296ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5de7f3a65f574e9ba8dd0e0d9e83a5f9'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:42.186583 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f8c128d1-5274-4fb9-b28e-55843ae296ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5de7f3a65f574e9ba8dd0e0d9e83a5f9'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:42.278863 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1316/2631] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:52:41 2022] DELETE /identity/v3/users/5de7f3a65f574e9ba8dd0e0d9e83a5f9 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:42.283472 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee575331-7f29-45e0-9e7c-3533c393d988 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:42.283964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee575331-7f29-45e0-9e7c-3533c393d988 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:42.284085 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee575331-7f29-45e0-9e7c-3533c393d988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:42.301317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-ee575331-7f29-45e0-9e7c-3533c393d988 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:42.302561 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-ee575331-7f29-45e0-9e7c-3533c393d988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:42.353113 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ebe1f-f412-48d9-a5e8-4f9afa59655b None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:42.353400 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ebe1f-f412-48d9-a5e8-4f9afa59655b None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:42.353672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ebe1f-f412-48d9-a5e8-4f9afa59655b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:42.366775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1316/2632] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:42.372668 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-452ebe1f-f412-48d9-a5e8-4f9afa59655b None None] MFA Rules not processed for user `58e1f1ac6ea14da0be6e9a24d007847a`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:42.373891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-452ebe1f-f412-48d9-a5e8-4f9afa59655b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:42.376973 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee575331-7f29-45e0-9e7c-3533c393d988 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:42.378622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-ee575331-7f29-45e0-9e7c-3533c393d988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:42.432871 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1317/2633] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Jun 27 18:52:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:42.440392 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee575331-7f29-45e0-9e7c-3533c393d988 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:42.444904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de673c17-05d5-480f-9960-c7311ad49e84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:42.445547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de673c17-05d5-480f-9960-c7311ad49e84 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:42.445838 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de673c17-05d5-480f-9960-c7311ad49e84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:42.446096 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de673c17-05d5-480f-9960-c7311ad49e84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:42.459523 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de673c17-05d5-480f-9960-c7311ad49e84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:42.460851 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de673c17-05d5-480f-9960-c7311ad49e84 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:42.471419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1317/2634] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:52:42.563724 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ebe1f-f412-48d9-a5e8-4f9afa59655b None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:42.565162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-452ebe1f-f412-48d9-a5e8-4f9afa59655b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:42.578670 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:42.634247 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ebe1f-f412-48d9-a5e8-4f9afa59655b None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:42.639898 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a72f0-4b26-4042-9b39-f07e888188a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1529c9a45434a19968e924797688f06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:42.640566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a72f0-4b26-4042-9b39-f07e888188a8 service swift] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:42.640848 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a72f0-4b26-4042-9b39-f07e888188a8 service swift] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:42.641124 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a72f0-4b26-4042-9b39-f07e888188a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:42.649933 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a72f0-4b26-4042-9b39-f07e888188a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:42.651640 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a72f0-4b26-4042-9b39-f07e888188a8 service swift] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:42.660843 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1318/2635] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Jun 27 18:52:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 18:52:42.932544 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de673c17-05d5-480f-9960-c7311ad49e84 service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:42.935247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-de673c17-05d5-480f-9960-c7311ad49e84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:42.952154 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de673c17-05d5-480f-9960-c7311ad49e84 service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:42.958262 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59198769-0c2d-48f2-a7ec-a3628fb6c2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:42.959183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59198769-0c2d-48f2-a7ec-a3628fb6c2f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:42.959527 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59198769-0c2d-48f2-a7ec-a3628fb6c2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:42.963450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59198769-0c2d-48f2-a7ec-a3628fb6c2f8 admin admin] PATH_INFO: `/v3/projects/124cd6c5f6df40c5b20167f6a49c917d` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:42.963450 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59198769-0c2d-48f2-a7ec-a3628fb6c2f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=124cd6c5f6df40c5b20167f6a49c917d)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:42.964143 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59198769-0c2d-48f2-a7ec-a3628fb6c2f8 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:43.080514 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-59198769-0c2d-48f2-a7ec-a3628fb6c2f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '124cd6c5f6df40c5b20167f6a49c917d', 'request_id': 'req-59198769-0c2d-48f2-a7ec-a3628fb6c2f8'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:43.158556 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-59198769-0c2d-48f2-a7ec-a3628fb6c2f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '124cd6c5f6df40c5b20167f6a49c917d', 'request_id': 'req-59198769-0c2d-48f2-a7ec-a3628fb6c2f8'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:43.166752 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-59198769-0c2d-48f2-a7ec-a3628fb6c2f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '124cd6c5f6df40c5b20167f6a49c917d', 'request_id': 'req-59198769-0c2d-48f2-a7ec-a3628fb6c2f8'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:43.177277 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-59198769-0c2d-48f2-a7ec-a3628fb6c2f8 admin admin] The token cache is being invalidate because project 124cd6c5f6df40c5b20167f6a49c917d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:52:43.177572 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-59198769-0c2d-48f2-a7ec-a3628fb6c2f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:43.178370 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-59198769-0c2d-48f2-a7ec-a3628fb6c2f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:43.180099 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1318/2636] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:52:42 2022] DELETE /identity/v3/projects/124cd6c5f6df40c5b20167f6a49c917d => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:43.184477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a72f0-4b26-4042-9b39-f07e888188a8 service swift] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:43.195047 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a72f0-4b26-4042-9b39-f07e888188a8 service swift] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:43.256615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dffdc-865f-4aa5-974e-e6b196fb9e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:43.257483 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dffdc-865f-4aa5-974e-e6b196fb9e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:43.257760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dffdc-865f-4aa5-974e-e6b196fb9e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:43.258036 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dffdc-865f-4aa5-974e-e6b196fb9e70 admin admin] PATH_INFO: `/v3/users/0c2ddb049ca34f5cb9e852e53ea776e0` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:43.259348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959dffdc-865f-4aa5-974e-e6b196fb9e70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c2ddb049ca34f5cb9e852e53ea776e0)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:43.262061 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959dffdc-865f-4aa5-974e-e6b196fb9e70 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:43.418391 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-959dffdc-865f-4aa5-974e-e6b196fb9e70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c2ddb049ca34f5cb9e852e53ea776e0'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:43.425172 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-959dffdc-865f-4aa5-974e-e6b196fb9e70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c2ddb049ca34f5cb9e852e53ea776e0'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:43.425852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-959dffdc-865f-4aa5-974e-e6b196fb9e70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c2ddb049ca34f5cb9e852e53ea776e0'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:43.426519 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-959dffdc-865f-4aa5-974e-e6b196fb9e70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c2ddb049ca34f5cb9e852e53ea776e0'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:43.443596 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-959dffdc-865f-4aa5-974e-e6b196fb9e70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c2ddb049ca34f5cb9e852e53ea776e0'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:43.480973 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233d558-864d-4f98-a18a-f3d7cc33eb56 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:43.481455 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233d558-864d-4f98-a18a-f3d7cc33eb56 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:43.481550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c233d558-864d-4f98-a18a-f3d7cc33eb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:43.487507 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:43.500786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-c233d558-864d-4f98-a18a-f3d7cc33eb56 None None] MFA Rules not processed for user `58e1f1ac6ea14da0be6e9a24d007847a`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:43.501980 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-c233d558-864d-4f98-a18a-f3d7cc33eb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:43.517868 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1319/2637] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:52:43 2022] DELETE /identity/v3/users/0c2ddb049ca34f5cb9e852e53ea776e0 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:43.563903 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1319/2638] 10.222.0.15 () {58 vars in 967 bytes} [Mon Jun 27 18:52:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:52:43.577561 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dffdc-865f-4aa5-974e-e6b196fb9e70 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:43.579162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-959dffdc-865f-4aa5-974e-e6b196fb9e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:43.601863 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:43.650012 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dffdc-865f-4aa5-974e-e6b196fb9e70 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:43.655117 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747d37f-392f-466b-861a-96cbbcaf8ed1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1529c9a45434a19968e924797688f06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:43.655759 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747d37f-392f-466b-861a-96cbbcaf8ed1 service swift] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:43.656041 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747d37f-392f-466b-861a-96cbbcaf8ed1 service swift] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:43.656319 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747d37f-392f-466b-861a-96cbbcaf8ed1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:43.720485 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747d37f-392f-466b-861a-96cbbcaf8ed1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:43.722075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747d37f-392f-466b-861a-96cbbcaf8ed1 service swift] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:43.733030 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1320/2639] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Jun 27 18:52:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 18:52:43.911837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233d558-864d-4f98-a18a-f3d7cc33eb56 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:43.913292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-c233d558-864d-4f98-a18a-f3d7cc33eb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:43.928666 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c233d558-864d-4f98-a18a-f3d7cc33eb56 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:43.989705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc5752-b576-4ccc-8bc0-7c087bb1858e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:43.990508 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc5752-b576-4ccc-8bc0-7c087bb1858e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:43.990793 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc5752-b576-4ccc-8bc0-7c087bb1858e admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:43.991068 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc5752-b576-4ccc-8bc0-7c087bb1858e admin admin] PATH_INFO: `/v3/projects/9ffe62941e94480cbcf2f90ef1318042` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:43.992253 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc5752-b576-4ccc-8bc0-7c087bb1858e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ffe62941e94480cbcf2f90ef1318042)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:43.995104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc5752-b576-4ccc-8bc0-7c087bb1858e admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:44.102022 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2edc5752-b576-4ccc-8bc0-7c087bb1858e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ffe62941e94480cbcf2f90ef1318042', 'request_id': 'req-2edc5752-b576-4ccc-8bc0-7c087bb1858e'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:44.164357 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2edc5752-b576-4ccc-8bc0-7c087bb1858e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ffe62941e94480cbcf2f90ef1318042', 'request_id': 'req-2edc5752-b576-4ccc-8bc0-7c087bb1858e'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:44.172388 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2edc5752-b576-4ccc-8bc0-7c087bb1858e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ffe62941e94480cbcf2f90ef1318042', 'request_id': 'req-2edc5752-b576-4ccc-8bc0-7c087bb1858e'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:44.196002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2edc5752-b576-4ccc-8bc0-7c087bb1858e admin admin] The token cache is being invalidate because project 9ffe62941e94480cbcf2f90ef1318042 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:52:44.196411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2edc5752-b576-4ccc-8bc0-7c087bb1858e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:44.199513 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2edc5752-b576-4ccc-8bc0-7c087bb1858e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:44.200890 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1320/2640] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:52:43 2022] DELETE /identity/v3/projects/9ffe62941e94480cbcf2f90ef1318042 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:44.205742 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605abd4-2268-46be-bbaf-ae31a6225c70 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:44.206635 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605abd4-2268-46be-bbaf-ae31a6225c70 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:44.207081 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605abd4-2268-46be-bbaf-ae31a6225c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:44.226793 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-d605abd4-2268-46be-bbaf-ae31a6225c70 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:44.228681 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-d605abd4-2268-46be-bbaf-ae31a6225c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:44.294419 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1321/2641] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:44.299290 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf99d2-134a-4eed-9370-6f3fa004684e None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:44.299580 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf99d2-134a-4eed-9370-6f3fa004684e None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:44.299859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf99d2-134a-4eed-9370-6f3fa004684e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:44.318398 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-adcf99d2-134a-4eed-9370-6f3fa004684e None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:44.319604 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-adcf99d2-134a-4eed-9370-6f3fa004684e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:44.327713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1321/2642] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:44.331856 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d605abd4-2268-46be-bbaf-ae31a6225c70 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:44.342108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d605abd4-2268-46be-bbaf-ae31a6225c70 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:44.346806 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b0cb4-c5ec-40fd-a627-9e126f23eb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:44.347452 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b0cb4-c5ec-40fd-a627-9e126f23eb21 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:44.347739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b0cb4-c5ec-40fd-a627-9e126f23eb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:44.348117 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b0cb4-c5ec-40fd-a627-9e126f23eb21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:44.348849 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b0cb4-c5ec-40fd-a627-9e126f23eb21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:52:44.349158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b0cb4-c5ec-40fd-a627-9e126f23eb21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:44.350793 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b0cb4-c5ec-40fd-a627-9e126f23eb21 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:44.362945 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1322/2643] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:52:44.367119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf99d2-134a-4eed-9370-6f3fa004684e None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:44.376719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf99d2-134a-4eed-9370-6f3fa004684e None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:44.381459 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641e346-fe6c-423c-a6b0-32b479a7e56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:44.382220 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641e346-fe6c-423c-a6b0-32b479a7e56f admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:44.382389 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641e346-fe6c-423c-a6b0-32b479a7e56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:44.382669 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641e346-fe6c-423c-a6b0-32b479a7e56f admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:44.383405 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f641e346-fe6c-423c-a6b0-32b479a7e56f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:44.387052 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f641e346-fe6c-423c-a6b0-32b479a7e56f admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:44.399297 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d7300-644a-4cde-9b25-0706cfd1c677 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:44.399586 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d7300-644a-4cde-9b25-0706cfd1c677 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:44.399895 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d7300-644a-4cde-9b25-0706cfd1c677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:44.423790 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-168d7300-644a-4cde-9b25-0706cfd1c677 None None] MFA Rules not processed for user `58e1f1ac6ea14da0be6e9a24d007847a`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:44.425002 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-168d7300-644a-4cde-9b25-0706cfd1c677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:44.471381 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:44.481471 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1322/2644] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:52:44 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:44.483444 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1323/2645] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Jun 27 18:52:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:52:44.485820 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641e346-fe6c-423c-a6b0-32b479a7e56f admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:44.496275 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d7300-644a-4cde-9b25-0706cfd1c677 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:44.496719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:44.497859 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-168d7300-644a-4cde-9b25-0706cfd1c677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:44.548761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641e346-fe6c-423c-a6b0-32b479a7e56f admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:44.553625 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0952f0-27d4-4a7f-9aa5-f02e59809d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:44.554279 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0952f0-27d4-4a7f-9aa5-f02e59809d37 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:44.554821 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0952f0-27d4-4a7f-9aa5-f02e59809d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:44.555097 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0952f0-27d4-4a7f-9aa5-f02e59809d37 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:44.555781 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0952f0-27d4-4a7f-9aa5-f02e59809d37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:44.557915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0952f0-27d4-4a7f-9aa5-f02e59809d37 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:44.573407 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d7300-644a-4cde-9b25-0706cfd1c677 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:44.627397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1323/2646] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:52:44 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:44.631404 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0952f0-27d4-4a7f-9aa5-f02e59809d37 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:44.636455 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183323ae-8658-4a1c-911b-7dc2177ca1a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1529c9a45434a19968e924797688f06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:44.637114 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183323ae-8658-4a1c-911b-7dc2177ca1a9 service swift] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:44.637395 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183323ae-8658-4a1c-911b-7dc2177ca1a9 service swift] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:44.637672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183323ae-8658-4a1c-911b-7dc2177ca1a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:44.642761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0952f0-27d4-4a7f-9aa5-f02e59809d37 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:44.647597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d441f-f43b-4fcc-aa96-155559c9651e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:44.648237 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d441f-f43b-4fcc-aa96-155559c9651e admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:44.648524 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d441f-f43b-4fcc-aa96-155559c9651e admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:44.648921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d441f-f43b-4fcc-aa96-155559c9651e admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:44.649639 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d441f-f43b-4fcc-aa96-155559c9651e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:44.652063 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d441f-f43b-4fcc-aa96-155559c9651e admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:44.662205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1324/2647] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:52:44.666329 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d441f-f43b-4fcc-aa96-155559c9651e admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:44.676656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d441f-f43b-4fcc-aa96-155559c9651e admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:44.683804 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf376f-0e0f-49d7-98de-1d3d4d5fa7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:44.684919 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf376f-0e0f-49d7-98de-1d3d4d5fa7a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:44.685622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf376f-0e0f-49d7-98de-1d3d4d5fa7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:44.686508 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf376f-0e0f-49d7-98de-1d3d4d5fa7a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:44.687218 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf376f-0e0f-49d7-98de-1d3d4d5fa7a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:44.689629 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf376f-0e0f-49d7-98de-1d3d4d5fa7a1 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:44.702499 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183323ae-8658-4a1c-911b-7dc2177ca1a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:44.704234 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183323ae-8658-4a1c-911b-7dc2177ca1a9 service swift] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:44.705917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1325/2648] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:52:44.706875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf376f-0e0f-49d7-98de-1d3d4d5fa7a1 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:44.714755 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1324/2649] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Jun 27 18:52:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 18:52:44.724410 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf376f-0e0f-49d7-98de-1d3d4d5fa7a1 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:44.729458 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d425a2b4-2f24-4c39-8e31-b967b6abe368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:44.730348 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425a2b4-2f24-4c39-8e31-b967b6abe368 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:44.730663 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425a2b4-2f24-4c39-8e31-b967b6abe368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:44.730940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425a2b4-2f24-4c39-8e31-b967b6abe368 admin admin] PATH_INFO: `/v3/projects/b48dac23399e4253ae9004198ec6b217/users/eeb9b5b7d2ba4bdf9eb0076d07ab17b4/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:44.749076 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d425a2b4-2f24-4c39-8e31-b967b6abe368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b48dac23399e4253ae9004198ec6b217, user_id=eeb9b5b7d2ba4bdf9eb0076d07ab17b4, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:44.751803 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d425a2b4-2f24-4c39-8e31-b967b6abe368 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:44.879696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1326/2650] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:44 2022] PUT /identity/v3/projects/b48dac23399e4253ae9004198ec6b217/users/eeb9b5b7d2ba4bdf9eb0076d07ab17b4/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:44.891214 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2dffd-ad8f-46eb-90f2-0b994aafe96e None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:44.891595 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2dffd-ad8f-46eb-90f2-0b994aafe96e None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:44.892050 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2dffd-ad8f-46eb-90f2-0b994aafe96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:44.918956 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-1fa2dffd-ad8f-46eb-90f2-0b994aafe96e None None] MFA Rules not processed for user `eeb9b5b7d2ba4bdf9eb0076d07ab17b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:44.920237 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-1fa2dffd-ad8f-46eb-90f2-0b994aafe96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:44.946157 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:44.994593 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1325/2651] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:52:44.999739 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d425a2b4-2f24-4c39-8e31-b967b6abe368 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:45.064648 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d425a2b4-2f24-4c39-8e31-b967b6abe368 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:45.069574 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbacbd-399f-421b-8889-0ebe6b25e9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:45.070267 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbacbd-399f-421b-8889-0ebe6b25e9f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:45.070603 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbacbd-399f-421b-8889-0ebe6b25e9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:45.070881 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbacbd-399f-421b-8889-0ebe6b25e9f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:45.071608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbacbd-399f-421b-8889-0ebe6b25e9f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:45.074129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbacbd-399f-421b-8889-0ebe6b25e9f8 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:45.146883 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1327/2652] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:52:44 2022] POST /identity/v3/projects => generated 377 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:45.148357 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2dffd-ad8f-46eb-90f2-0b994aafe96e None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:45.159478 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2dffd-ad8f-46eb-90f2-0b994aafe96e None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:45.219656 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d150b-6323-4862-ad8c-bb479a046507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:45.220395 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d150b-6323-4862-ad8c-bb479a046507 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:45.220685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d150b-6323-4862-ad8c-bb479a046507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:45.220960 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d150b-6323-4862-ad8c-bb479a046507 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:45.221669 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d150b-6323-4862-ad8c-bb479a046507 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:45.223769 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d150b-6323-4862-ad8c-bb479a046507 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:45.323935 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1326/2653] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:52:45 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:45.326812 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbacbd-399f-421b-8889-0ebe6b25e9f8 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:45.354695 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbacbd-399f-421b-8889-0ebe6b25e9f8 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:45.362179 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab762f-2060-45e8-b0fc-5e98e0a046fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:45.363309 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab762f-2060-45e8-b0fc-5e98e0a046fc admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:45.363925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab762f-2060-45e8-b0fc-5e98e0a046fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:45.366558 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab762f-2060-45e8-b0fc-5e98e0a046fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:45.369164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab762f-2060-45e8-b0fc-5e98e0a046fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:45.371716 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab762f-2060-45e8-b0fc-5e98e0a046fc admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:45.382624 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c051794-26fd-4d3c-884f-3ca4fe9da71e None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:45.383404 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c051794-26fd-4d3c-884f-3ca4fe9da71e None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:45.383895 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c051794-26fd-4d3c-884f-3ca4fe9da71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:45.389612 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1328/2654] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:52:45.395385 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab762f-2060-45e8-b0fc-5e98e0a046fc admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:45.407083 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab762f-2060-45e8-b0fc-5e98e0a046fc admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:45.408761 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-8c051794-26fd-4d3c-884f-3ca4fe9da71e None None] MFA Rules not processed for user `58e1f1ac6ea14da0be6e9a24d007847a`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:45.410144 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-8c051794-26fd-4d3c-884f-3ca4fe9da71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:45.412859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd16269-6927-415c-b0cc-1d84eb2ed3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:45.413549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd16269-6927-415c-b0cc-1d84eb2ed3b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:45.413827 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd16269-6927-415c-b0cc-1d84eb2ed3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:45.414103 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd16269-6927-415c-b0cc-1d84eb2ed3b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:45.414855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd16269-6927-415c-b0cc-1d84eb2ed3b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:45.417142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd16269-6927-415c-b0cc-1d84eb2ed3b2 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:45.428492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1329/2655] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:45.432729 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd16269-6927-415c-b0cc-1d84eb2ed3b2 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:45.442835 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd16269-6927-415c-b0cc-1d84eb2ed3b2 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:45.448844 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884bf42-7213-4e80-b349-76abc9476c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:45.449996 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884bf42-7213-4e80-b349-76abc9476c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:45.450280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884bf42-7213-4e80-b349-76abc9476c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:45.450591 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884bf42-7213-4e80-b349-76abc9476c17 admin admin] PATH_INFO: `/v3/projects/f69e26431c9749f4aee64b45b723a454/users/e343e7d97e58424fb134ebfd27bf6472/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:45.465622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:45.469942 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884bf42-7213-4e80-b349-76abc9476c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f69e26431c9749f4aee64b45b723a454, user_id=e343e7d97e58424fb134ebfd27bf6472, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:45.473973 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884bf42-7213-4e80-b349-76abc9476c17 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:45.476016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1327/2656] 10.222.0.15 () {58 vars in 967 bytes} [Mon Jun 27 18:52:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:45.489285 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c051794-26fd-4d3c-884f-3ca4fe9da71e None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:45.554633 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:45.559762 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c051794-26fd-4d3c-884f-3ca4fe9da71e None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:45.565227 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a46a3c-3dd9-4d18-9901-6528392db2a5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1529c9a45434a19968e924797688f06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:45.565885 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a46a3c-3dd9-4d18-9901-6528392db2a5 service swift] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:45.566172 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a46a3c-3dd9-4d18-9901-6528392db2a5 service swift] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:45.566480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a46a3c-3dd9-4d18-9901-6528392db2a5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:45.575784 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a46a3c-3dd9-4d18-9901-6528392db2a5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:45.577302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a46a3c-3dd9-4d18-9901-6528392db2a5 service swift] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:45.581822 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1330/2657] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:45 2022] PUT /identity/v3/projects/f69e26431c9749f4aee64b45b723a454/users/e343e7d97e58424fb134ebfd27bf6472/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:52:45.590450 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1328/2658] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Jun 27 18:52:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 27 18:52:45.592829 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ce9af-9a0e-4ae4-ba57-6f1ad9ceee2f None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:45.593037 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ce9af-9a0e-4ae4-ba57-6f1ad9ceee2f None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:45.593348 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ce9af-9a0e-4ae4-ba57-6f1ad9ceee2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:45.621504 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-c00ce9af-9a0e-4ae4-ba57-6f1ad9ceee2f None None] MFA Rules not processed for user `e343e7d97e58424fb134ebfd27bf6472`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:45.622755 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-c00ce9af-9a0e-4ae4-ba57-6f1ad9ceee2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:45.688098 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1331/2659] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:45.692937 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a46a3c-3dd9-4d18-9901-6528392db2a5 service swift] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:45.704340 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a46a3c-3dd9-4d18-9901-6528392db2a5 service swift] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:45.762615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63aa528-963b-4a7d-990b-7da95dc9be33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:45.763410 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63aa528-963b-4a7d-990b-7da95dc9be33 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:45.763696 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63aa528-963b-4a7d-990b-7da95dc9be33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:45.763971 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63aa528-963b-4a7d-990b-7da95dc9be33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:45.764692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63aa528-963b-4a7d-990b-7da95dc9be33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:45.766723 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63aa528-963b-4a7d-990b-7da95dc9be33 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:45.858463 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1329/2660] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:52:45 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:52:45.862676 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ce9af-9a0e-4ae4-ba57-6f1ad9ceee2f None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:45.874045 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ce9af-9a0e-4ae4-ba57-6f1ad9ceee2f None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:45.935003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87518b0-61da-4e77-a242-d56613e30208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:45.935679 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87518b0-61da-4e77-a242-d56613e30208 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:45.935964 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87518b0-61da-4e77-a242-d56613e30208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:45.936243 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87518b0-61da-4e77-a242-d56613e30208 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:45.936929 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87518b0-61da-4e77-a242-d56613e30208 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:45.938900 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87518b0-61da-4e77-a242-d56613e30208 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:46.022527 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1332/2661] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:52:45 2022] POST /identity/v3/users => generated 296 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:46.030619 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63aa528-963b-4a7d-990b-7da95dc9be33 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:46.042486 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63aa528-963b-4a7d-990b-7da95dc9be33 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:46.047458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7b870-4714-48fa-95c1-4168921460cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:46.048119 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7b870-4714-48fa-95c1-4168921460cb admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:46.048432 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7b870-4714-48fa-95c1-4168921460cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:46.048709 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7b870-4714-48fa-95c1-4168921460cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:46.049431 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7b870-4714-48fa-95c1-4168921460cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:46.051062 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7b870-4714-48fa-95c1-4168921460cb admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:46.062634 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1330/2662] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:46.067492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87518b0-61da-4e77-a242-d56613e30208 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:46.078580 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87518b0-61da-4e77-a242-d56613e30208 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:46.083619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5106c-98c0-4ae2-95f2-fc3ca399da44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:46.084543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5106c-98c0-4ae2-95f2-fc3ca399da44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:46.084805 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5106c-98c0-4ae2-95f2-fc3ca399da44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:46.085114 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5106c-98c0-4ae2-95f2-fc3ca399da44 admin admin] PATH_INFO: `/v3/projects/195c7e78fe3b4abd9773ad0ab36c0b06/users/6d6b0d9d70104055866391663370dda1/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:46.103621 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5106c-98c0-4ae2-95f2-fc3ca399da44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=195c7e78fe3b4abd9773ad0ab36c0b06, user_id=6d6b0d9d70104055866391663370dda1, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:46.107527 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5106c-98c0-4ae2-95f2-fc3ca399da44 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:46.171308 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1333/2663] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:46 2022] PUT /identity/v3/projects/195c7e78fe3b4abd9773ad0ab36c0b06/users/6d6b0d9d70104055866391663370dda1/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:46.175430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7b870-4714-48fa-95c1-4168921460cb admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:46.186364 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7b870-4714-48fa-95c1-4168921460cb admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:46.245277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028bfe48-f0a1-453b-95c2-63fa98116517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:46.245932 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bfe48-f0a1-453b-95c2-63fa98116517 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:46.246207 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bfe48-f0a1-453b-95c2-63fa98116517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:46.246523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bfe48-f0a1-453b-95c2-63fa98116517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:46.247247 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028bfe48-f0a1-453b-95c2-63fa98116517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:46.248847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028bfe48-f0a1-453b-95c2-63fa98116517 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:46.260645 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1331/2664] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:46.264948 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5106c-98c0-4ae2-95f2-fc3ca399da44 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:46.276269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5106c-98c0-4ae2-95f2-fc3ca399da44 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:46.282093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d6e6b-0033-45fb-a77b-fb8865ae7539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:46.283032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d6e6b-0033-45fb-a77b-fb8865ae7539 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:46.283311 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d6e6b-0033-45fb-a77b-fb8865ae7539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:46.283620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d6e6b-0033-45fb-a77b-fb8865ae7539 admin admin] PATH_INFO: `/v3/projects/195c7e78fe3b4abd9773ad0ab36c0b06/users/6d6b0d9d70104055866391663370dda1/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:46.285568 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d6e6b-0033-45fb-a77b-fb8865ae7539 admin admin] RBAC: Authorizing `identity:create_grant(project_id=195c7e78fe3b4abd9773ad0ab36c0b06, user_id=6d6b0d9d70104055866391663370dda1, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:46.288203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d6e6b-0033-45fb-a77b-fb8865ae7539 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:46.299995 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1334/2665] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:46 2022] PUT /identity/v3/projects/195c7e78fe3b4abd9773ad0ab36c0b06/users/6d6b0d9d70104055866391663370dda1/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:52:46.310553 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4399455-6b50-4b8b-9c56-bc792e9f7747 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:46.315031 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4399455-6b50-4b8b-9c56-bc792e9f7747 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:46.315031 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4399455-6b50-4b8b-9c56-bc792e9f7747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:46.337231 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-b4399455-6b50-4b8b-9c56-bc792e9f7747 None None] MFA Rules not processed for user `6d6b0d9d70104055866391663370dda1`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:46.339194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-b4399455-6b50-4b8b-9c56-bc792e9f7747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:46.391489 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:46.414596 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1332/2666] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:46.424606 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d6e6b-0033-45fb-a77b-fb8865ae7539 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:46.426368 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-1d3d6e6b-0033-45fb-a77b-fb8865ae7539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:46.464391 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:46.485640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d6e6b-0033-45fb-a77b-fb8865ae7539 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:46.489331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2670c-91b0-4415-81c2-8ec4a2d66818 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:46.489973 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2670c-91b0-4415-81c2-8ec4a2d66818 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:46.490297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2670c-91b0-4415-81c2-8ec4a2d66818 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:46.490601 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2670c-91b0-4415-81c2-8ec4a2d66818 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:46.500538 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2670c-91b0-4415-81c2-8ec4a2d66818 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:46.501949 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2670c-91b0-4415-81c2-8ec4a2d66818 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:46.512761 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1335/2667] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:52:46.648128 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4399455-6b50-4b8b-9c56-bc792e9f7747 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:46.660722 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4399455-6b50-4b8b-9c56-bc792e9f7747 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:46.664689 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc61bc-9f91-4e9c-b30a-446ce82d597b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:46.665419 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc61bc-9f91-4e9c-b30a-446ce82d597b service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:46.666117 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc61bc-9f91-4e9c-b30a-446ce82d597b service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:46.666399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc61bc-9f91-4e9c-b30a-446ce82d597b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:46.755620 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc61bc-9f91-4e9c-b30a-446ce82d597b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:46.757127 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc61bc-9f91-4e9c-b30a-446ce82d597b service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:46.767111 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1333/2668] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:52:48.198853 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2670c-91b0-4415-81c2-8ec4a2d66818 service cinder] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:48.200357 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-6ef2670c-91b0-4415-81c2-8ec4a2d66818 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:48.266864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2670c-91b0-4415-81c2-8ec4a2d66818 service cinder] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:48.271433 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f0beae-2217-48bf-8354-834b85e7c1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:48.272446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f0beae-2217-48bf-8354-834b85e7c1a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:48.272734 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f0beae-2217-48bf-8354-834b85e7c1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:48.273036 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f0beae-2217-48bf-8354-834b85e7c1a8 admin admin] PATH_INFO: `/v3/users/aa14260afbce48ffbb851ff7065aea64` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:48.274239 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f0beae-2217-48bf-8354-834b85e7c1a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa14260afbce48ffbb851ff7065aea64)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:48.276889 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f0beae-2217-48bf-8354-834b85e7c1a8 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:48.440674 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-42f0beae-2217-48bf-8354-834b85e7c1a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa14260afbce48ffbb851ff7065aea64'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:48.448272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-42f0beae-2217-48bf-8354-834b85e7c1a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa14260afbce48ffbb851ff7065aea64'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:48.449000 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-42f0beae-2217-48bf-8354-834b85e7c1a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa14260afbce48ffbb851ff7065aea64'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:48.460699 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-42f0beae-2217-48bf-8354-834b85e7c1a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa14260afbce48ffbb851ff7065aea64'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:48.518231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-42f0beae-2217-48bf-8354-834b85e7c1a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa14260afbce48ffbb851ff7065aea64'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:48.524462 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1336/2669] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:52:48 2022] DELETE /identity/v3/users/aa14260afbce48ffbb851ff7065aea64 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:52:48.529117 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d062fb4-d453-4f31-afc4-b4270c091e36 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:48.529614 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d062fb4-d453-4f31-afc4-b4270c091e36 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:48.529715 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d062fb4-d453-4f31-afc4-b4270c091e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:48.546374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-0d062fb4-d453-4f31-afc4-b4270c091e36 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:48.547591 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-0d062fb4-d453-4f31-afc4-b4270c091e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:48.606981 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1334/2670] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:48.616624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f0beae-2217-48bf-8354-834b85e7c1a8 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:48.618298 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-42f0beae-2217-48bf-8354-834b85e7c1a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:48.675629 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f0beae-2217-48bf-8354-834b85e7c1a8 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:48.679288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbac26d-cd7a-44e1-a95e-f7528c3fa293 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:48.679925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbac26d-cd7a-44e1-a95e-f7528c3fa293 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:48.680205 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbac26d-cd7a-44e1-a95e-f7528c3fa293 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:48.680483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbac26d-cd7a-44e1-a95e-f7528c3fa293 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:48.690241 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbac26d-cd7a-44e1-a95e-f7528c3fa293 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:48.691766 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbac26d-cd7a-44e1-a95e-f7528c3fa293 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:48.696620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:48.706067 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1337/2671] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:52:48.980024 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d062fb4-d453-4f31-afc4-b4270c091e36 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:48.981675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-0d062fb4-d453-4f31-afc4-b4270c091e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:48.995992 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d062fb4-d453-4f31-afc4-b4270c091e36 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:49.001600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f94d07-0cec-479c-b2e3-6a1ec6e6c570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:49.011747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f94d07-0cec-479c-b2e3-6a1ec6e6c570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:49.011747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f94d07-0cec-479c-b2e3-6a1ec6e6c570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:49.011747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f94d07-0cec-479c-b2e3-6a1ec6e6c570 admin admin] PATH_INFO: `/v3/projects/ac1847cdd8cc43d9b34490bd5e871e2b` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:49.011747 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f94d07-0cec-479c-b2e3-6a1ec6e6c570 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac1847cdd8cc43d9b34490bd5e871e2b)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:49.012854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f94d07-0cec-479c-b2e3-6a1ec6e6c570 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:49.130115 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-66f94d07-0cec-479c-b2e3-6a1ec6e6c570 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac1847cdd8cc43d9b34490bd5e871e2b', 'request_id': 'req-66f94d07-0cec-479c-b2e3-6a1ec6e6c570'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:49.237600 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-66f94d07-0cec-479c-b2e3-6a1ec6e6c570 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac1847cdd8cc43d9b34490bd5e871e2b', 'request_id': 'req-66f94d07-0cec-479c-b2e3-6a1ec6e6c570'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:49.245909 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-66f94d07-0cec-479c-b2e3-6a1ec6e6c570 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac1847cdd8cc43d9b34490bd5e871e2b', 'request_id': 'req-66f94d07-0cec-479c-b2e3-6a1ec6e6c570'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:49.256627 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-66f94d07-0cec-479c-b2e3-6a1ec6e6c570 admin admin] The token cache is being invalidate because project ac1847cdd8cc43d9b34490bd5e871e2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:52:49.256918 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-66f94d07-0cec-479c-b2e3-6a1ec6e6c570 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:49.257559 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-66f94d07-0cec-479c-b2e3-6a1ec6e6c570 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:49.259185 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1335/2672] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:52:48 2022] DELETE /identity/v3/projects/ac1847cdd8cc43d9b34490bd5e871e2b => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:49.264266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8345ae-3d5f-42df-8888-a622cb6adfda None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:49.270590 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8345ae-3d5f-42df-8888-a622cb6adfda None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:49.270590 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8345ae-3d5f-42df-8888-a622cb6adfda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:49.285890 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-de8345ae-3d5f-42df-8888-a622cb6adfda None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:49.287113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-de8345ae-3d5f-42df-8888-a622cb6adfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:49.345999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1338/2673] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:52:49.354831 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015db880-d4e2-44f1-af6c-dadf6e013c97 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:49.355123 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015db880-d4e2-44f1-af6c-dadf6e013c97 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:49.355406 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015db880-d4e2-44f1-af6c-dadf6e013c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:49.375308 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-015db880-d4e2-44f1-af6c-dadf6e013c97 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:49.376515 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-015db880-d4e2-44f1-af6c-dadf6e013c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:49.385066 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1336/2674] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:49.389049 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8345ae-3d5f-42df-8888-a622cb6adfda None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:49.398787 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8345ae-3d5f-42df-8888-a622cb6adfda None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:49.404562 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87cc417-509c-4641-94d6-bf83325a3bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:49.405220 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87cc417-509c-4641-94d6-bf83325a3bda admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:49.405503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87cc417-509c-4641-94d6-bf83325a3bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:49.405783 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87cc417-509c-4641-94d6-bf83325a3bda admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:49.406548 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87cc417-509c-4641-94d6-bf83325a3bda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:52:49.406858 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87cc417-509c-4641-94d6-bf83325a3bda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:49.408484 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87cc417-509c-4641-94d6-bf83325a3bda admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:49.421336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1339/2675] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:52:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:52:49.425732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015db880-d4e2-44f1-af6c-dadf6e013c97 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:49.430184 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:49.435471 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015db880-d4e2-44f1-af6c-dadf6e013c97 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:49.440408 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ba01c-8ef4-4d71-8643-2ceab566dd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:49.441060 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ba01c-8ef4-4d71-8643-2ceab566dd05 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:49.441331 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ba01c-8ef4-4d71-8643-2ceab566dd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:49.441603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ba01c-8ef4-4d71-8643-2ceab566dd05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:49.442283 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ba01c-8ef4-4d71-8643-2ceab566dd05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:49.444268 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ba01c-8ef4-4d71-8643-2ceab566dd05 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:49.566190 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1337/2676] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:52:49 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:49.570151 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87cc417-509c-4641-94d6-bf83325a3bda admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:49.581317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87cc417-509c-4641-94d6-bf83325a3bda admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:49.637845 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c4eb8-96db-4fbc-b08d-50b00d671e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:49.638544 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c4eb8-96db-4fbc-b08d-50b00d671e5b admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:49.638827 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c4eb8-96db-4fbc-b08d-50b00d671e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:49.639105 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c4eb8-96db-4fbc-b08d-50b00d671e5b admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:49.639799 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c4eb8-96db-4fbc-b08d-50b00d671e5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:49.641780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c4eb8-96db-4fbc-b08d-50b00d671e5b admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:49.725617 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1340/2677] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:52:49 2022] POST /identity/v3/users => generated 292 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:49.729781 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ba01c-8ef4-4d71-8643-2ceab566dd05 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:49.740714 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ba01c-8ef4-4d71-8643-2ceab566dd05 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:49.746061 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4e339-511d-4683-977b-1ee2b63c9711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:49.747974 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4e339-511d-4683-977b-1ee2b63c9711 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:49.747974 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4e339-511d-4683-977b-1ee2b63c9711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:49.747974 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4e339-511d-4683-977b-1ee2b63c9711 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:49.748862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4e339-511d-4683-977b-1ee2b63c9711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:49.749760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4e339-511d-4683-977b-1ee2b63c9711 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:49.761329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1338/2678] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:49.769042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c4eb8-96db-4fbc-b08d-50b00d671e5b admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:49.780010 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c4eb8-96db-4fbc-b08d-50b00d671e5b admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:49.784757 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd1c04-4a38-4364-a3cf-5a919bd0e596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:49.785423 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd1c04-4a38-4364-a3cf-5a919bd0e596 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:49.785705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd1c04-4a38-4364-a3cf-5a919bd0e596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:49.785981 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd1c04-4a38-4364-a3cf-5a919bd0e596 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:49.786845 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd1c04-4a38-4364-a3cf-5a919bd0e596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:49.788495 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd1c04-4a38-4364-a3cf-5a919bd0e596 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:49.798743 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1341/2679] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:49.803199 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4e339-511d-4683-977b-1ee2b63c9711 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:49.813095 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4e339-511d-4683-977b-1ee2b63c9711 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:49.817621 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476e5f7-84cc-4a39-8c6f-1c67e9340f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:49.818506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476e5f7-84cc-4a39-8c6f-1c67e9340f60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:49.818790 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476e5f7-84cc-4a39-8c6f-1c67e9340f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:49.819062 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476e5f7-84cc-4a39-8c6f-1c67e9340f60 admin admin] PATH_INFO: `/v3/projects/22ff7f833172448fa47fbb796cd91c3d/users/5d58cb1968f240d693e4c414869ffe44/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:49.833692 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476e5f7-84cc-4a39-8c6f-1c67e9340f60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22ff7f833172448fa47fbb796cd91c3d, user_id=5d58cb1968f240d693e4c414869ffe44, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:49.836361 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476e5f7-84cc-4a39-8c6f-1c67e9340f60 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:49.924396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1339/2680] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:49 2022] PUT /identity/v3/projects/22ff7f833172448fa47fbb796cd91c3d/users/5d58cb1968f240d693e4c414869ffe44/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:49.929943 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fe5a7-7857-49ad-a7d1-37703e43da6c None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:49.930258 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fe5a7-7857-49ad-a7d1-37703e43da6c None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:49.930634 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fe5a7-7857-49ad-a7d1-37703e43da6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:49.949115 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-1c5fe5a7-7857-49ad-a7d1-37703e43da6c None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:49.957341 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-1c5fe5a7-7857-49ad-a7d1-37703e43da6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:50.011403 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1342/2681] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:50.020991 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476e5f7-84cc-4a39-8c6f-1c67e9340f60 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:50.022683 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-4476e5f7-84cc-4a39-8c6f-1c67e9340f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:50.083662 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476e5f7-84cc-4a39-8c6f-1c67e9340f60 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:50.087397 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ee1c3-9b57-4525-8340-e658124681f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:50.088088 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ee1c3-9b57-4525-8340-e658124681f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:50.088381 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ee1c3-9b57-4525-8340-e658124681f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:50.088654 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ee1c3-9b57-4525-8340-e658124681f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:50.098159 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ee1c3-9b57-4525-8340-e658124681f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:50.099671 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ee1c3-9b57-4525-8340-e658124681f2 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:50.110132 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1340/2682] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:52:50.941061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5fe5a7-7857-49ad-a7d1-37703e43da6c None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:50.942602 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-1c5fe5a7-7857-49ad-a7d1-37703e43da6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:50.952650 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:50.956380 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5fe5a7-7857-49ad-a7d1-37703e43da6c None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:50.961898 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b16fb7-1399-4827-a9a4-e4ecef94db63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:50.963799 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b16fb7-1399-4827-a9a4-e4ecef94db63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:50.964124 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b16fb7-1399-4827-a9a4-e4ecef94db63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:50.964397 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b16fb7-1399-4827-a9a4-e4ecef94db63 admin admin] PATH_INFO: `/v3/users/eeb9b5b7d2ba4bdf9eb0076d07ab17b4` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:50.965622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b16fb7-1399-4827-a9a4-e4ecef94db63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeb9b5b7d2ba4bdf9eb0076d07ab17b4)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:50.967568 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b16fb7-1399-4827-a9a4-e4ecef94db63 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:51.134071 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-40b16fb7-1399-4827-a9a4-e4ecef94db63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeb9b5b7d2ba4bdf9eb0076d07ab17b4'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:51.148361 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-40b16fb7-1399-4827-a9a4-e4ecef94db63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeb9b5b7d2ba4bdf9eb0076d07ab17b4'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:51.153575 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-40b16fb7-1399-4827-a9a4-e4ecef94db63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeb9b5b7d2ba4bdf9eb0076d07ab17b4'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:51.166398 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-40b16fb7-1399-4827-a9a4-e4ecef94db63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeb9b5b7d2ba4bdf9eb0076d07ab17b4'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:51.210022 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-40b16fb7-1399-4827-a9a4-e4ecef94db63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeb9b5b7d2ba4bdf9eb0076d07ab17b4'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:51.217700 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1343/2683] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:52:50 2022] DELETE /identity/v3/users/eeb9b5b7d2ba4bdf9eb0076d07ab17b4 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:51.222198 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d23816-8b90-4a2d-84cb-0d8745526a47 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:51.228759 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d23816-8b90-4a2d-84cb-0d8745526a47 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:51.228759 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d23816-8b90-4a2d-84cb-0d8745526a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:51.240571 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-d6d23816-8b90-4a2d-84cb-0d8745526a47 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:51.241768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-d6d23816-8b90-4a2d-84cb-0d8745526a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:51.312695 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1341/2684] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:51.323575 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b16fb7-1399-4827-a9a4-e4ecef94db63 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:51.325091 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-40b16fb7-1399-4827-a9a4-e4ecef94db63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:51.398471 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b16fb7-1399-4827-a9a4-e4ecef94db63 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:51.404791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486609ae-8358-4a1b-9d0e-6b3d627fc8eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:51.405574 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486609ae-8358-4a1b-9d0e-6b3d627fc8eb service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:51.405863 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486609ae-8358-4a1b-9d0e-6b3d627fc8eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:51.406143 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486609ae-8358-4a1b-9d0e-6b3d627fc8eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:51.420982 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486609ae-8358-4a1b-9d0e-6b3d627fc8eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:51.424555 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486609ae-8358-4a1b-9d0e-6b3d627fc8eb service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:51.433384 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1344/2685] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:52:52.091058 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d23816-8b90-4a2d-84cb-0d8745526a47 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:52.093299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-d6d23816-8b90-4a2d-84cb-0d8745526a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:52.115112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d23816-8b90-4a2d-84cb-0d8745526a47 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:52.124326 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad76f4-1884-4d16-87fb-be05d2e22263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:52.125222 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad76f4-1884-4d16-87fb-be05d2e22263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:52.125505 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad76f4-1884-4d16-87fb-be05d2e22263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:52.125780 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad76f4-1884-4d16-87fb-be05d2e22263 admin admin] PATH_INFO: `/v3/projects/b48dac23399e4253ae9004198ec6b217` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:52.127336 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ad76f4-1884-4d16-87fb-be05d2e22263 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b48dac23399e4253ae9004198ec6b217)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:52.129301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ad76f4-1884-4d16-87fb-be05d2e22263 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:52.257232 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:52.261451 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-34ad76f4-1884-4d16-87fb-be05d2e22263 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b48dac23399e4253ae9004198ec6b217', 'request_id': 'req-34ad76f4-1884-4d16-87fb-be05d2e22263'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:52.347803 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-34ad76f4-1884-4d16-87fb-be05d2e22263 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b48dac23399e4253ae9004198ec6b217', 'request_id': 'req-34ad76f4-1884-4d16-87fb-be05d2e22263'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:52.355051 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-34ad76f4-1884-4d16-87fb-be05d2e22263 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b48dac23399e4253ae9004198ec6b217', 'request_id': 'req-34ad76f4-1884-4d16-87fb-be05d2e22263'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:52.366244 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-34ad76f4-1884-4d16-87fb-be05d2e22263 admin admin] The token cache is being invalidate because project b48dac23399e4253ae9004198ec6b217 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:52:52.366590 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-34ad76f4-1884-4d16-87fb-be05d2e22263 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:52.367250 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-34ad76f4-1884-4d16-87fb-be05d2e22263 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:52.370164 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1342/2686] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:52:52 2022] DELETE /identity/v3/projects/b48dac23399e4253ae9004198ec6b217 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:52.373937 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486609ae-8358-4a1b-9d0e-6b3d627fc8eb service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:52.386829 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486609ae-8358-4a1b-9d0e-6b3d627fc8eb service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:52.463129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2763b-933d-4211-8a8b-798922a6715b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:52.464025 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2763b-933d-4211-8a8b-798922a6715b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:52.464332 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2763b-933d-4211-8a8b-798922a6715b admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:52.464747 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2763b-933d-4211-8a8b-798922a6715b admin admin] PATH_INFO: `/v3/users/e343e7d97e58424fb134ebfd27bf6472` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:52.466275 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2763b-933d-4211-8a8b-798922a6715b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e343e7d97e58424fb134ebfd27bf6472)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:52.468579 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2763b-933d-4211-8a8b-798922a6715b admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:52.500972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:52.659652 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-7ee2763b-933d-4211-8a8b-798922a6715b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e343e7d97e58424fb134ebfd27bf6472'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:52.670012 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-7ee2763b-933d-4211-8a8b-798922a6715b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e343e7d97e58424fb134ebfd27bf6472'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:52.671292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-7ee2763b-933d-4211-8a8b-798922a6715b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e343e7d97e58424fb134ebfd27bf6472'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:52.684909 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-7ee2763b-933d-4211-8a8b-798922a6715b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e343e7d97e58424fb134ebfd27bf6472'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:52.776215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-7ee2763b-933d-4211-8a8b-798922a6715b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e343e7d97e58424fb134ebfd27bf6472'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:52.786637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1345/2687] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:52:52 2022] DELETE /identity/v3/users/e343e7d97e58424fb134ebfd27bf6472 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:53.643571 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad76f4-1884-4d16-87fb-be05d2e22263 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:53.645726 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-34ad76f4-1884-4d16-87fb-be05d2e22263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:53.659880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad76f4-1884-4d16-87fb-be05d2e22263 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:53.739695 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69e400-d21e-4016-a8de-58cb50540261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:53.743633 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69e400-d21e-4016-a8de-58cb50540261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:53.743633 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69e400-d21e-4016-a8de-58cb50540261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:53.743633 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69e400-d21e-4016-a8de-58cb50540261 admin admin] PATH_INFO: `/v3/projects/f69e26431c9749f4aee64b45b723a454` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:53.745691 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d69e400-d21e-4016-a8de-58cb50540261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f69e26431c9749f4aee64b45b723a454)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:53.747709 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d69e400-d21e-4016-a8de-58cb50540261 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:53.955356 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7d69e400-d21e-4016-a8de-58cb50540261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f69e26431c9749f4aee64b45b723a454', 'request_id': 'req-7d69e400-d21e-4016-a8de-58cb50540261'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:54.045698 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7d69e400-d21e-4016-a8de-58cb50540261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f69e26431c9749f4aee64b45b723a454', 'request_id': 'req-7d69e400-d21e-4016-a8de-58cb50540261'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:54.053262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7d69e400-d21e-4016-a8de-58cb50540261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f69e26431c9749f4aee64b45b723a454', 'request_id': 'req-7d69e400-d21e-4016-a8de-58cb50540261'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:54.066995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7d69e400-d21e-4016-a8de-58cb50540261 admin admin] The token cache is being invalidate because project f69e26431c9749f4aee64b45b723a454 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:52:54.067417 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7d69e400-d21e-4016-a8de-58cb50540261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:54.068552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7d69e400-d21e-4016-a8de-58cb50540261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:54.069803 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1343/2688] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:52:53 2022] DELETE /identity/v3/projects/f69e26431c9749f4aee64b45b723a454 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:54.074849 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2763b-933d-4211-8a8b-798922a6715b admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:54.080183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-7ee2763b-933d-4211-8a8b-798922a6715b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:54.092289 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2763b-933d-4211-8a8b-798922a6715b admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:54.168696 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3565bf93-115a-46cd-8039-a0d00041164b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:54.169583 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565bf93-115a-46cd-8039-a0d00041164b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:54.169864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565bf93-115a-46cd-8039-a0d00041164b admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:54.169864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565bf93-115a-46cd-8039-a0d00041164b admin admin] PATH_INFO: `/v3/users/6d6b0d9d70104055866391663370dda1` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:54.171796 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3565bf93-115a-46cd-8039-a0d00041164b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d6b0d9d70104055866391663370dda1)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:54.173367 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3565bf93-115a-46cd-8039-a0d00041164b admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:54.365942 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-3565bf93-115a-46cd-8039-a0d00041164b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d6b0d9d70104055866391663370dda1'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:54.376791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-3565bf93-115a-46cd-8039-a0d00041164b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d6b0d9d70104055866391663370dda1'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:54.377553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-3565bf93-115a-46cd-8039-a0d00041164b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d6b0d9d70104055866391663370dda1'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:54.398804 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-3565bf93-115a-46cd-8039-a0d00041164b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d6b0d9d70104055866391663370dda1'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:54.479457 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-3565bf93-115a-46cd-8039-a0d00041164b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d6b0d9d70104055866391663370dda1'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:54.488703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1346/2689] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:52:54 2022] DELETE /identity/v3/users/6d6b0d9d70104055866391663370dda1 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:52:55.060960 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69e400-d21e-4016-a8de-58cb50540261 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:55.062535 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-7d69e400-d21e-4016-a8de-58cb50540261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:55.083206 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69e400-d21e-4016-a8de-58cb50540261 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:55.160768 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff0dfd-a5e7-4a9c-b36b-b4a8789e2ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:55.161549 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff0dfd-a5e7-4a9c-b36b-b4a8789e2ac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:55.161866 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff0dfd-a5e7-4a9c-b36b-b4a8789e2ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:55.162111 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff0dfd-a5e7-4a9c-b36b-b4a8789e2ac7 admin admin] PATH_INFO: `/v3/projects/195c7e78fe3b4abd9773ad0ab36c0b06` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:55.163396 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff0dfd-a5e7-4a9c-b36b-b4a8789e2ac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=195c7e78fe3b4abd9773ad0ab36c0b06)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:55.166371 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff0dfd-a5e7-4a9c-b36b-b4a8789e2ac7 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:55.169904 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:55.302577 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f7ff0dfd-a5e7-4a9c-b36b-b4a8789e2ac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '195c7e78fe3b4abd9773ad0ab36c0b06', 'request_id': 'req-f7ff0dfd-a5e7-4a9c-b36b-b4a8789e2ac7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:55.427870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f7ff0dfd-a5e7-4a9c-b36b-b4a8789e2ac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '195c7e78fe3b4abd9773ad0ab36c0b06', 'request_id': 'req-f7ff0dfd-a5e7-4a9c-b36b-b4a8789e2ac7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:55.436748 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f7ff0dfd-a5e7-4a9c-b36b-b4a8789e2ac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '195c7e78fe3b4abd9773ad0ab36c0b06', 'request_id': 'req-f7ff0dfd-a5e7-4a9c-b36b-b4a8789e2ac7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:55.449072 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f7ff0dfd-a5e7-4a9c-b36b-b4a8789e2ac7 admin admin] The token cache is being invalidate because project 195c7e78fe3b4abd9773ad0ab36c0b06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:52:55.454348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f7ff0dfd-a5e7-4a9c-b36b-b4a8789e2ac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:55.454348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f7ff0dfd-a5e7-4a9c-b36b-b4a8789e2ac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:52:55.454348 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1344/2690] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:52:55 2022] DELETE /identity/v3/projects/195c7e78fe3b4abd9773ad0ab36c0b06 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:55.457665 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d636e3-e9bc-453f-96b8-58044fb8d86e None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:55.458131 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d636e3-e9bc-453f-96b8-58044fb8d86e None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:55.458414 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d636e3-e9bc-453f-96b8-58044fb8d86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:55.477632 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-f0d636e3-e9bc-453f-96b8-58044fb8d86e None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:55.479021 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f0d636e3-e9bc-453f-96b8-58044fb8d86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:55.492035 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:55.561281 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1347/2691] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:55.570053 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5197964-e988-43af-9f74-8b030e0c7d12 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:55.570633 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5197964-e988-43af-9f74-8b030e0c7d12 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:55.571101 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5197964-e988-43af-9f74-8b030e0c7d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:55.605810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-f5197964-e988-43af-9f74-8b030e0c7d12 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:55.607068 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-f5197964-e988-43af-9f74-8b030e0c7d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:55.622606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1345/2692] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:55.625351 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d636e3-e9bc-453f-96b8-58044fb8d86e None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:55.641019 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d636e3-e9bc-453f-96b8-58044fb8d86e None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:55.647396 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d8699-7c16-4631-8488-9a3cc576b16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:55.652855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d8699-7c16-4631-8488-9a3cc576b16c admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:55.652855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d8699-7c16-4631-8488-9a3cc576b16c admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:55.652855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d8699-7c16-4631-8488-9a3cc576b16c admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:55.652855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d8699-7c16-4631-8488-9a3cc576b16c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:52:55.652855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d8699-7c16-4631-8488-9a3cc576b16c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:55.652855 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d8699-7c16-4631-8488-9a3cc576b16c admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:55.667027 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1348/2693] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:52:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:52:55.676065 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5197964-e988-43af-9f74-8b030e0c7d12 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:55.683601 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5197964-e988-43af-9f74-8b030e0c7d12 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:55.688471 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922702e6-036d-4cdd-84c1-b3cf3310fcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:55.689097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922702e6-036d-4cdd-84c1-b3cf3310fcb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:55.689378 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922702e6-036d-4cdd-84c1-b3cf3310fcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:55.689653 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922702e6-036d-4cdd-84c1-b3cf3310fcb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:55.690340 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922702e6-036d-4cdd-84c1-b3cf3310fcb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:55.692270 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922702e6-036d-4cdd-84c1-b3cf3310fcb4 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:55.776074 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1346/2694] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:52:55 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:55.777514 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d8699-7c16-4631-8488-9a3cc576b16c admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:55.792875 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d8699-7c16-4631-8488-9a3cc576b16c admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:55.872303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0336510-60c4-4d6d-bad5-5ba4c8fb3bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:55.873728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0336510-60c4-4d6d-bad5-5ba4c8fb3bcc admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:55.873728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0336510-60c4-4d6d-bad5-5ba4c8fb3bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:55.873728 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0336510-60c4-4d6d-bad5-5ba4c8fb3bcc admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:55.875239 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0336510-60c4-4d6d-bad5-5ba4c8fb3bcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:55.878007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0336510-60c4-4d6d-bad5-5ba4c8fb3bcc admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:55.966035 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1349/2695] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:52:55 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:55.967767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922702e6-036d-4cdd-84c1-b3cf3310fcb4 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:55.978488 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922702e6-036d-4cdd-84c1-b3cf3310fcb4 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:55.983324 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad536b4-05a2-43eb-96c6-fc80a1a6adae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:55.984057 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad536b4-05a2-43eb-96c6-fc80a1a6adae admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:55.984430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad536b4-05a2-43eb-96c6-fc80a1a6adae admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:55.984777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad536b4-05a2-43eb-96c6-fc80a1a6adae admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:55.985573 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad536b4-05a2-43eb-96c6-fc80a1a6adae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:55.987306 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad536b4-05a2-43eb-96c6-fc80a1a6adae admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:55.999852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1347/2696] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:56.003988 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0336510-60c4-4d6d-bad5-5ba4c8fb3bcc admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:56.015681 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0336510-60c4-4d6d-bad5-5ba4c8fb3bcc admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:56.020342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6ad25-e364-4c4f-bd38-adb312c21150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:56.020992 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6ad25-e364-4c4f-bd38-adb312c21150 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:56.021270 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6ad25-e364-4c4f-bd38-adb312c21150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:56.021543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6ad25-e364-4c4f-bd38-adb312c21150 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:56.022407 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6ad25-e364-4c4f-bd38-adb312c21150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:56.023995 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6ad25-e364-4c4f-bd38-adb312c21150 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:56.034321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1350/2697] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:56.038478 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad536b4-05a2-43eb-96c6-fc80a1a6adae admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:56.065602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad536b4-05a2-43eb-96c6-fc80a1a6adae admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:56.076418 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0fa7a-8e9d-4e77-be3e-fd76cba6075a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:56.078715 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0fa7a-8e9d-4e77-be3e-fd76cba6075a admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:56.079000 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0fa7a-8e9d-4e77-be3e-fd76cba6075a admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:56.079277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0fa7a-8e9d-4e77-be3e-fd76cba6075a admin admin] PATH_INFO: `/v3/projects/07079a8e7b0d4ff0a0f4cf3276d3c5d3/users/0b2eeeea8176450bbdc87985c331bca3/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:56.102552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0fa7a-8e9d-4e77-be3e-fd76cba6075a admin admin] RBAC: Authorizing `identity:create_grant(project_id=07079a8e7b0d4ff0a0f4cf3276d3c5d3, user_id=0b2eeeea8176450bbdc87985c331bca3, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:56.105830 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d0fa7a-8e9d-4e77-be3e-fd76cba6075a admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:56.165151 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1348/2698] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:56 2022] PUT /identity/v3/projects/07079a8e7b0d4ff0a0f4cf3276d3c5d3/users/0b2eeeea8176450bbdc87985c331bca3/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:56.180342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a13e12-735a-4889-9f26-e74fe1835b9e None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:56.192093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a13e12-735a-4889-9f26-e74fe1835b9e None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:56.192093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a13e12-735a-4889-9f26-e74fe1835b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:56.215322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-02a13e12-735a-4889-9f26-e74fe1835b9e None None] MFA Rules not processed for user `0b2eeeea8176450bbdc87985c331bca3`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:56.215322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-02a13e12-735a-4889-9f26-e74fe1835b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:56.329135 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1351/2699] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:56.344493 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0fa7a-8e9d-4e77-be3e-fd76cba6075a admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:56.349439 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-e4d0fa7a-8e9d-4e77-be3e-fd76cba6075a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:56.421100 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0fa7a-8e9d-4e77-be3e-fd76cba6075a admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:56.427219 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83639ea5-2be8-4148-a18e-a43d6df8b0ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:56.428166 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83639ea5-2be8-4148-a18e-a43d6df8b0ea service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:56.428631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83639ea5-2be8-4148-a18e-a43d6df8b0ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:56.429082 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83639ea5-2be8-4148-a18e-a43d6df8b0ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:56.449373 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83639ea5-2be8-4148-a18e-a43d6df8b0ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:56.450948 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83639ea5-2be8-4148-a18e-a43d6df8b0ea service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:56.462785 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1349/2700] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:52:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:52:56.552640 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c3ad1-582f-4bb9-a827-c4c98334b363 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:56.553184 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c3ad1-582f-4bb9-a827-c4c98334b363 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:56.553184 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c3ad1-582f-4bb9-a827-c4c98334b363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:56.591613 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:56.596019 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-031c3ad1-582f-4bb9-a827-c4c98334b363 None None] MFA Rules not processed for user `5d58cb1968f240d693e4c414869ffe44`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:52:56.597568 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-031c3ad1-582f-4bb9-a827-c4c98334b363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:56.671685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1352/2701] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:52:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:52:56.676277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83639ea5-2be8-4148-a18e-a43d6df8b0ea service cinder] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:56.681981 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-83639ea5-2be8-4148-a18e-a43d6df8b0ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:52:56.716042 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:52:56.750435 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83639ea5-2be8-4148-a18e-a43d6df8b0ea service cinder] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:56.755308 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c6fe4-6cd3-4bb8-9779-25e7f6ada4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:56.755995 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c6fe4-6cd3-4bb8-9779-25e7f6ada4c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:56.756282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c6fe4-6cd3-4bb8-9779-25e7f6ada4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:56.756560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c6fe4-6cd3-4bb8-9779-25e7f6ada4c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:56.757338 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c6fe4-6cd3-4bb8-9779-25e7f6ada4c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:56.759384 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c6fe4-6cd3-4bb8-9779-25e7f6ada4c8 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:56.847450 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1350/2702] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:52:56 2022] POST /identity/v3/projects => generated 381 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:56.851109 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031c3ad1-582f-4bb9-a827-c4c98334b363 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:56.862507 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031c3ad1-582f-4bb9-a827-c4c98334b363 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:56.928051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3e231-a882-472a-88b3-2b721fc00dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:56.928051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3e231-a882-472a-88b3-2b721fc00dcc admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:56.928051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3e231-a882-472a-88b3-2b721fc00dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:56.928051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf3e231-a882-472a-88b3-2b721fc00dcc admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:56.928694 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf3e231-a882-472a-88b3-2b721fc00dcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:56.929203 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf3e231-a882-472a-88b3-2b721fc00dcc admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:57.002643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1353/2703] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:52:56 2022] POST /identity/v3/users => generated 297 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:52:57.004198 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c6fe4-6cd3-4bb8-9779-25e7f6ada4c8 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:57.015270 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c6fe4-6cd3-4bb8-9779-25e7f6ada4c8 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:57.019874 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfe967-3052-43ba-a6b9-872167aaa154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:57.020526 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfe967-3052-43ba-a6b9-872167aaa154 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:57.020814 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfe967-3052-43ba-a6b9-872167aaa154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:57.021091 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfe967-3052-43ba-a6b9-872167aaa154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:57.021834 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcfe967-3052-43ba-a6b9-872167aaa154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:57.023484 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcfe967-3052-43ba-a6b9-872167aaa154 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:57.034082 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1351/2704] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:57.038247 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3e231-a882-472a-88b3-2b721fc00dcc admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:57.048366 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf3e231-a882-472a-88b3-2b721fc00dcc admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:57.053443 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378835a4-91cb-4511-a5f8-b065097f4aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:57.054350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378835a4-91cb-4511-a5f8-b065097f4aa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:57.054664 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378835a4-91cb-4511-a5f8-b065097f4aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:57.054962 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378835a4-91cb-4511-a5f8-b065097f4aa9 admin admin] PATH_INFO: `/v3/projects/7e62b539f1fa447794a1ff02384d28cd/users/b3e359535c0546d1897d9addb5fae985/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:57.072904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378835a4-91cb-4511-a5f8-b065097f4aa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e62b539f1fa447794a1ff02384d28cd, user_id=b3e359535c0546d1897d9addb5fae985, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:57.075606 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378835a4-91cb-4511-a5f8-b065097f4aa9 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:57.164701 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1354/2705] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:57 2022] PUT /identity/v3/projects/7e62b539f1fa447794a1ff02384d28cd/users/b3e359535c0546d1897d9addb5fae985/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:52:57.169479 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfe967-3052-43ba-a6b9-872167aaa154 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:57.179885 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfe967-3052-43ba-a6b9-872167aaa154 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:57.238958 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50026d4-ba4a-42c8-b3f7-1d660312de56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:57.239730 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50026d4-ba4a-42c8-b3f7-1d660312de56 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:57.240014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50026d4-ba4a-42c8-b3f7-1d660312de56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:57.240290 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50026d4-ba4a-42c8-b3f7-1d660312de56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:57.241018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50026d4-ba4a-42c8-b3f7-1d660312de56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:57.242630 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50026d4-ba4a-42c8-b3f7-1d660312de56 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:57.253304 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1352/2706] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:52:57.257398 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378835a4-91cb-4511-a5f8-b065097f4aa9 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:52:57.268532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378835a4-91cb-4511-a5f8-b065097f4aa9 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:52:57.275929 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d68de9-eeb9-44a0-8eec-466f2a972ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:52:57.275929 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d68de9-eeb9-44a0-8eec-466f2a972ed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:52:57.275929 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d68de9-eeb9-44a0-8eec-466f2a972ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:52:57.275929 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d68de9-eeb9-44a0-8eec-466f2a972ed2 admin admin] PATH_INFO: `/v3/projects/7e62b539f1fa447794a1ff02384d28cd/users/b3e359535c0546d1897d9addb5fae985/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:52:57.277321 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d68de9-eeb9-44a0-8eec-466f2a972ed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e62b539f1fa447794a1ff02384d28cd, user_id=b3e359535c0546d1897d9addb5fae985, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:52:57.279970 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d68de9-eeb9-44a0-8eec-466f2a972ed2 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:52:57.290360 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1355/2707] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:52:57 2022] PUT /identity/v3/projects/7e62b539f1fa447794a1ff02384d28cd/users/b3e359535c0546d1897d9addb5fae985/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:53:00.768343 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50026d4-ba4a-42c8-b3f7-1d660312de56 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:00.779786 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50026d4-ba4a-42c8-b3f7-1d660312de56 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:00.856592 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa5346-7c19-47f6-b777-034fb6b36999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:00.857566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa5346-7c19-47f6-b777-034fb6b36999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:00.857923 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa5346-7c19-47f6-b777-034fb6b36999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:00.858272 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa5346-7c19-47f6-b777-034fb6b36999 admin admin] PATH_INFO: `/v3/users/0b2eeeea8176450bbdc87985c331bca3` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:00.859682 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa5346-7c19-47f6-b777-034fb6b36999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b2eeeea8176450bbdc87985c331bca3)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:00.861715 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafa5346-7c19-47f6-b777-034fb6b36999 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:01.068030 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:53:01.071109 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-bafa5346-7c19-47f6-b777-034fb6b36999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b2eeeea8176450bbdc87985c331bca3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:53:01.080577 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-bafa5346-7c19-47f6-b777-034fb6b36999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b2eeeea8176450bbdc87985c331bca3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:53:01.081729 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-bafa5346-7c19-47f6-b777-034fb6b36999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b2eeeea8176450bbdc87985c331bca3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:53:01.082374 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-bafa5346-7c19-47f6-b777-034fb6b36999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b2eeeea8176450bbdc87985c331bca3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:53:01.096495 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-bafa5346-7c19-47f6-b777-034fb6b36999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b2eeeea8176450bbdc87985c331bca3'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:53:01.178542 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1353/2708] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:53:00 2022] DELETE /identity/v3/users/0b2eeeea8176450bbdc87985c331bca3 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:53:01.181403 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13eb45a-3a81-4b02-b8dc-92c04a1bb6e5 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:01.189698 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13eb45a-3a81-4b02-b8dc-92c04a1bb6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:01.189698 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13eb45a-3a81-4b02-b8dc-92c04a1bb6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:01.205961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-f13eb45a-3a81-4b02-b8dc-92c04a1bb6e5 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:53:01.207639 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f13eb45a-3a81-4b02-b8dc-92c04a1bb6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:01.280651 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:53:01.305896 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1356/2709] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:53:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:53:01.313733 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa5346-7c19-47f6-b777-034fb6b36999 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:01.315234 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-bafa5346-7c19-47f6-b777-034fb6b36999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:01.414894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa5346-7c19-47f6-b777-034fb6b36999 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:01.419898 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02626b0f-ce06-498d-bb66-f87b14c67b0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:01.424724 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02626b0f-ce06-498d-bb66-f87b14c67b0e service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:01.424724 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02626b0f-ce06-498d-bb66-f87b14c67b0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:01.424724 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02626b0f-ce06-498d-bb66-f87b14c67b0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:01.434477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02626b0f-ce06-498d-bb66-f87b14c67b0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:01.436265 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02626b0f-ce06-498d-bb66-f87b14c67b0e service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:01.449322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1354/2710] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:53:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:53:01.919780 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13eb45a-3a81-4b02-b8dc-92c04a1bb6e5 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:01.921861 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f13eb45a-3a81-4b02-b8dc-92c04a1bb6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:01.941427 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13eb45a-3a81-4b02-b8dc-92c04a1bb6e5 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:01.956909 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31abe1-886e-41a4-a705-6e7899c52a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:01.958453 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31abe1-886e-41a4-a705-6e7899c52a47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:01.959682 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31abe1-886e-41a4-a705-6e7899c52a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:01.960513 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31abe1-886e-41a4-a705-6e7899c52a47 admin admin] PATH_INFO: `/v3/projects/07079a8e7b0d4ff0a0f4cf3276d3c5d3` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:01.962999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31abe1-886e-41a4-a705-6e7899c52a47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07079a8e7b0d4ff0a0f4cf3276d3c5d3)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:01.968198 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31abe1-886e-41a4-a705-6e7899c52a47 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:02.182928 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-5c31abe1-886e-41a4-a705-6e7899c52a47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07079a8e7b0d4ff0a0f4cf3276d3c5d3', 'request_id': 'req-5c31abe1-886e-41a4-a705-6e7899c52a47'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:53:02.289498 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-5c31abe1-886e-41a4-a705-6e7899c52a47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07079a8e7b0d4ff0a0f4cf3276d3c5d3', 'request_id': 'req-5c31abe1-886e-41a4-a705-6e7899c52a47'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:53:02.308789 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-5c31abe1-886e-41a4-a705-6e7899c52a47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07079a8e7b0d4ff0a0f4cf3276d3c5d3', 'request_id': 'req-5c31abe1-886e-41a4-a705-6e7899c52a47'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:53:02.323066 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-5c31abe1-886e-41a4-a705-6e7899c52a47 admin admin] The token cache is being invalidate because project 07079a8e7b0d4ff0a0f4cf3276d3c5d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:53:02.323888 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-5c31abe1-886e-41a4-a705-6e7899c52a47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:53:02.324837 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-5c31abe1-886e-41a4-a705-6e7899c52a47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:53:02.326645 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1357/2711] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:53:01 2022] DELETE /identity/v3/projects/07079a8e7b0d4ff0a0f4cf3276d3c5d3 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:53:02.333194 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80b80f-b74f-4591-bd17-a078aceac9ec None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:02.333715 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80b80f-b74f-4591-bd17-a078aceac9ec None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:02.333804 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80b80f-b74f-4591-bd17-a078aceac9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:02.355683 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-8d80b80f-b74f-4591-bd17-a078aceac9ec None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:53:02.357604 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-8d80b80f-b74f-4591-bd17-a078aceac9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:02.422167 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1355/2712] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:53:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:53:02.430197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b0f38-b428-4b87-9527-3a8860cfa695 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:02.430596 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b0f38-b428-4b87-9527-3a8860cfa695 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:02.431974 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b0f38-b428-4b87-9527-3a8860cfa695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:02.451113 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-8e3b0f38-b428-4b87-9527-3a8860cfa695 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:53:02.452361 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-8e3b0f38-b428-4b87-9527-3a8860cfa695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:02.467485 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1358/2713] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:53:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:53:02.467746 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80b80f-b74f-4591-bd17-a078aceac9ec None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:02.479540 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80b80f-b74f-4591-bd17-a078aceac9ec None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:02.485232 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1782e64-b4d4-4bd3-bbda-76a82f8d2976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:02.485912 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1782e64-b4d4-4bd3-bbda-76a82f8d2976 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:02.490413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1782e64-b4d4-4bd3-bbda-76a82f8d2976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:02.490413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1782e64-b4d4-4bd3-bbda-76a82f8d2976 admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:02.490413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1782e64-b4d4-4bd3-bbda-76a82f8d2976 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:53:02.490413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1782e64-b4d4-4bd3-bbda-76a82f8d2976 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:02.490413 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1782e64-b4d4-4bd3-bbda-76a82f8d2976 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:02.509024 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1356/2714] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:53:02.512185 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b0f38-b428-4b87-9527-3a8860cfa695 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:02.523379 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b0f38-b428-4b87-9527-3a8860cfa695 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:02.529816 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d09fb4-0cf5-4e6b-9393-0b962bc13080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:02.530677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d09fb4-0cf5-4e6b-9393-0b962bc13080 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:02.531068 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d09fb4-0cf5-4e6b-9393-0b962bc13080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:02.531445 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d09fb4-0cf5-4e6b-9393-0b962bc13080 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:02.532331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d09fb4-0cf5-4e6b-9393-0b962bc13080 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:02.534485 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d09fb4-0cf5-4e6b-9393-0b962bc13080 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:02.636081 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1359/2715] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:53:02 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:53:02.639968 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1782e64-b4d4-4bd3-bbda-76a82f8d2976 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:02.651880 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1782e64-b4d4-4bd3-bbda-76a82f8d2976 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:02.737074 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:53:02.751136 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf975e7-0e9c-4081-93f6-5a42519bea2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:02.752289 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf975e7-0e9c-4081-93f6-5a42519bea2c admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:02.752851 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf975e7-0e9c-4081-93f6-5a42519bea2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:02.753355 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf975e7-0e9c-4081-93f6-5a42519bea2c admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:02.756176 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf975e7-0e9c-4081-93f6-5a42519bea2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:02.757550 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf975e7-0e9c-4081-93f6-5a42519bea2c admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:02.848360 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1357/2716] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:53:02 2022] POST /identity/v3/users => generated 289 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:53:02.852132 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d09fb4-0cf5-4e6b-9393-0b962bc13080 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:02.863791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d09fb4-0cf5-4e6b-9393-0b962bc13080 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:02.869441 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09d61c-f7cc-4fc1-a21d-69cd543c063b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:02.870549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09d61c-f7cc-4fc1-a21d-69cd543c063b admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:02.870841 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09d61c-f7cc-4fc1-a21d-69cd543c063b admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:02.871120 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09d61c-f7cc-4fc1-a21d-69cd543c063b admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:02.871876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b09d61c-f7cc-4fc1-a21d-69cd543c063b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:02.873527 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b09d61c-f7cc-4fc1-a21d-69cd543c063b admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:02.888057 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1360/2717] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:53:02.889079 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf975e7-0e9c-4081-93f6-5a42519bea2c admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:02.899455 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf975e7-0e9c-4081-93f6-5a42519bea2c admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:02.904734 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e0832-9e48-4399-b3ea-80e0cba78d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:02.905473 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e0832-9e48-4399-b3ea-80e0cba78d0f admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:02.905665 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e0832-9e48-4399-b3ea-80e0cba78d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:02.906168 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e0832-9e48-4399-b3ea-80e0cba78d0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:02.906757 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e0832-9e48-4399-b3ea-80e0cba78d0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:02.908394 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e0832-9e48-4399-b3ea-80e0cba78d0f admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:02.922204 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1358/2718] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:53:02.936765 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09d61c-f7cc-4fc1-a21d-69cd543c063b admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:02.949380 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09d61c-f7cc-4fc1-a21d-69cd543c063b admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:02.954362 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530a442-1f0e-4f87-9403-b3656c8c0aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:02.955541 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530a442-1f0e-4f87-9403-b3656c8c0aaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:02.955891 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530a442-1f0e-4f87-9403-b3656c8c0aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:02.956239 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530a442-1f0e-4f87-9403-b3656c8c0aaa admin admin] PATH_INFO: `/v3/projects/302e8c5b4c1f422fbd5cf5602a094d2e/users/b10a90cce1b048d7bd55938811301ab4/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:02.995419 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530a442-1f0e-4f87-9403-b3656c8c0aaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=302e8c5b4c1f422fbd5cf5602a094d2e, user_id=b10a90cce1b048d7bd55938811301ab4, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:03.001266 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98719ee4-76be-4f88-abec-45364495b051 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:03.002431 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98719ee4-76be-4f88-abec-45364495b051 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:03.002878 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530a442-1f0e-4f87-9403-b3656c8c0aaa admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:03.003042 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98719ee4-76be-4f88-abec-45364495b051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:03.032881 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-98719ee4-76be-4f88-abec-45364495b051 None None] MFA Rules not processed for user `b3e359535c0546d1897d9addb5fae985`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:53:03.034189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-98719ee4-76be-4f88-abec-45364495b051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:03.116468 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1359/2719] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:53:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:53:03.121712 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1361/2720] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:53:02 2022] PUT /identity/v3/projects/302e8c5b4c1f422fbd5cf5602a094d2e/users/b10a90cce1b048d7bd55938811301ab4/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:53:03.128301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f0d3d-7c2b-4b4a-9d0b-f9d2de81a0f6 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:03.128822 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f0d3d-7c2b-4b4a-9d0b-f9d2de81a0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:03.129255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f0d3d-7c2b-4b4a-9d0b-f9d2de81a0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:03.133292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530a442-1f0e-4f87-9403-b3656c8c0aaa admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:03.136526 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f530a442-1f0e-4f87-9403-b3656c8c0aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:03.149118 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-ed1f0d3d-7c2b-4b4a-9d0b-f9d2de81a0f6 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:53:03.151136 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ed1f0d3d-7c2b-4b4a-9d0b-f9d2de81a0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:03.208366 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530a442-1f0e-4f87-9403-b3656c8c0aaa admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:03.212348 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70d33a-b52a-45df-8c92-189cc424ceaf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75e75e5ae3ca45d0b503f6e52f399558', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:03.213719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70d33a-b52a-45df-8c92-189cc424ceaf service glance] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:03.214012 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70d33a-b52a-45df-8c92-189cc424ceaf service glance] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:03.214289 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70d33a-b52a-45df-8c92-189cc424ceaf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:03.216175 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-0c70d33a-b52a-45df-8c92-189cc424ceaf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:03.219459 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1360/2721] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:53:03.226713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f0d3d-7c2b-4b4a-9d0b-f9d2de81a0f6 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:03.229433 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ed1f0d3d-7c2b-4b4a-9d0b-f9d2de81a0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:03.251610 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:53:03.284955 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70d33a-b52a-45df-8c92-189cc424ceaf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:03.286974 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70d33a-b52a-45df-8c92-189cc424ceaf service glance] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:03.297071 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f0d3d-7c2b-4b4a-9d0b-f9d2de81a0f6 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:03.299274 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1362/2722] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:53:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:53:03.302988 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a079c-b4c4-4d6e-9e9d-3c5abdf82144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:03.303826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a079c-b4c4-4d6e-9e9d-3c5abdf82144 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:03.304104 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a079c-b4c4-4d6e-9e9d-3c5abdf82144 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:03.304408 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a079c-b4c4-4d6e-9e9d-3c5abdf82144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:03.315912 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a079c-b4c4-4d6e-9e9d-3c5abdf82144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:03.317799 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a079c-b4c4-4d6e-9e9d-3c5abdf82144 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:03.322703 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:53:03.329656 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1361/2723] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:53:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:53:03.584138 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f6327-21bd-4a13-a003-77ccdc9f1782 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:03.584428 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f6327-21bd-4a13-a003-77ccdc9f1782 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:03.584709 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f6327-21bd-4a13-a003-77ccdc9f1782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:03.602994 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-ca8f6327-21bd-4a13-a003-77ccdc9f1782 None None] MFA Rules not processed for user `58e1f1ac6ea14da0be6e9a24d007847a`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:53:03.604588 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-ca8f6327-21bd-4a13-a003-77ccdc9f1782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:03.674729 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1363/2724] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Jun 27 18:53:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:53:03.696116 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a079c-b4c4-4d6e-9e9d-3c5abdf82144 service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:03.698269 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-404a079c-b4c4-4d6e-9e9d-3c5abdf82144 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:03.787286 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a079c-b4c4-4d6e-9e9d-3c5abdf82144 service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:03.793089 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74061f4f-be78-4440-a77e-079f6954e37f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1529c9a45434a19968e924797688f06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:03.793663 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74061f4f-be78-4440-a77e-079f6954e37f service swift] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:03.793928 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74061f4f-be78-4440-a77e-079f6954e37f service swift] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:03.794271 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74061f4f-be78-4440-a77e-079f6954e37f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:03.806767 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74061f4f-be78-4440-a77e-079f6954e37f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:03.809274 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74061f4f-be78-4440-a77e-079f6954e37f service swift] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:03.823450 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1362/2725] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Jun 27 18:53:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 18:53:10.401865 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6246a9-91d6-44b3-98cd-05d95420d22f None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:10.402334 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6246a9-91d6-44b3-98cd-05d95420d22f None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:10.402575 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6246a9-91d6-44b3-98cd-05d95420d22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:10.441799 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-4d6246a9-91d6-44b3-98cd-05d95420d22f None None] MFA Rules not processed for user `b10a90cce1b048d7bd55938811301ab4`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:53:10.443730 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-4d6246a9-91d6-44b3-98cd-05d95420d22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:10.582492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1364/2726] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:53:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:53:10.585235 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74061f4f-be78-4440-a77e-079f6954e37f service swift] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:10.600088 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74061f4f-be78-4440-a77e-079f6954e37f service swift] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:10.605443 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f495a-1712-433f-9c0f-11f0c38df3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:10.606163 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f495a-1712-433f-9c0f-11f0c38df3da admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:10.606477 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f495a-1712-433f-9c0f-11f0c38df3da admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:10.606754 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f495a-1712-433f-9c0f-11f0c38df3da admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:10.607450 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f495a-1712-433f-9c0f-11f0c38df3da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:10.609448 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313f495a-1712-433f-9c0f-11f0c38df3da admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:10.701058 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1363/2727] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:53:10 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:53:10.708598 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6246a9-91d6-44b3-98cd-05d95420d22f None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:10.725815 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6246a9-91d6-44b3-98cd-05d95420d22f None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:10.835635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746b745-f3c1-4df8-bbec-d4780d342f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:10.836422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746b745-f3c1-4df8-bbec-d4780d342f1d admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:10.836700 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746b745-f3c1-4df8-bbec-d4780d342f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:10.836971 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746b745-f3c1-4df8-bbec-d4780d342f1d admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:10.837809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a746b745-f3c1-4df8-bbec-d4780d342f1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:10.840296 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a746b745-f3c1-4df8-bbec-d4780d342f1d admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:10.861670 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:53:10.965252 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1365/2728] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:53:10 2022] POST /identity/v3/users => generated 295 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:53:10.966517 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f495a-1712-433f-9c0f-11f0c38df3da admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:10.979555 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f495a-1712-433f-9c0f-11f0c38df3da admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:10.984889 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436b3c8-853c-4c16-bb29-fb4467ac16d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:10.985655 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436b3c8-853c-4c16-bb29-fb4467ac16d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:10.985939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436b3c8-853c-4c16-bb29-fb4467ac16d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:10.986218 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436b3c8-853c-4c16-bb29-fb4467ac16d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:10.986985 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436b3c8-853c-4c16-bb29-fb4467ac16d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:10.988603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436b3c8-853c-4c16-bb29-fb4467ac16d4 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:11.007754 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1364/2729] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:53:11.011919 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746b745-f3c1-4df8-bbec-d4780d342f1d admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:11.024600 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746b745-f3c1-4df8-bbec-d4780d342f1d admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:11.030442 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b53ce-b585-4d6d-823c-67ab1c003a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:11.031422 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b53ce-b585-4d6d-823c-67ab1c003a38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:11.032061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b53ce-b585-4d6d-823c-67ab1c003a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:11.032061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b53ce-b585-4d6d-823c-67ab1c003a38 admin admin] PATH_INFO: `/v3/projects/7f7af5ede07d48e3a9c650602a70db45/users/a70f695bbb7b494ab85f253ad5285e5a/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:11.049337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b53ce-b585-4d6d-823c-67ab1c003a38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f7af5ede07d48e3a9c650602a70db45, user_id=a70f695bbb7b494ab85f253ad5285e5a, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:11.052058 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b53ce-b585-4d6d-823c-67ab1c003a38 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:11.123512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1366/2730] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:53:11 2022] PUT /identity/v3/projects/7f7af5ede07d48e3a9c650602a70db45/users/a70f695bbb7b494ab85f253ad5285e5a/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:53:11.129177 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436b3c8-853c-4c16-bb29-fb4467ac16d4 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:11.145685 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436b3c8-853c-4c16-bb29-fb4467ac16d4 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:11.232040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b18da-2221-4732-826d-5361c69aa35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:11.233079 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b18da-2221-4732-826d-5361c69aa35f admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:11.233363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b18da-2221-4732-826d-5361c69aa35f admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:11.233641 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b18da-2221-4732-826d-5361c69aa35f admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:11.234377 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b18da-2221-4732-826d-5361c69aa35f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:11.236014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b18da-2221-4732-826d-5361c69aa35f admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:11.249913 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1365/2731] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:53:11.253266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b53ce-b585-4d6d-823c-67ab1c003a38 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:11.263120 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b53ce-b585-4d6d-823c-67ab1c003a38 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:11.270250 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c0cd5-42e9-482d-9870-701b5bed18bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:11.271219 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c0cd5-42e9-482d-9870-701b5bed18bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:11.271501 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c0cd5-42e9-482d-9870-701b5bed18bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:11.271773 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c0cd5-42e9-482d-9870-701b5bed18bd admin admin] PATH_INFO: `/v3/projects/7f7af5ede07d48e3a9c650602a70db45/users/a70f695bbb7b494ab85f253ad5285e5a/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:11.274605 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c0cd5-42e9-482d-9870-701b5bed18bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f7af5ede07d48e3a9c650602a70db45, user_id=a70f695bbb7b494ab85f253ad5285e5a, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:11.281042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c0cd5-42e9-482d-9870-701b5bed18bd admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:11.302968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1367/2732] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:53:11 2022] PUT /identity/v3/projects/7f7af5ede07d48e3a9c650602a70db45/users/a70f695bbb7b494ab85f253ad5285e5a/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:53:18.202439 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8ed8e-a456-4902-a178-79a0ac87d67c None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:18.203037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8ed8e-a456-4902-a178-79a0ac87d67c None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:18.203037 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8ed8e-a456-4902-a178-79a0ac87d67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:18.230917 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-0fa8ed8e-a456-4902-a178-79a0ac87d67c None None] MFA Rules not processed for user `5d58cb1968f240d693e4c414869ffe44`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:53:18.232169 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-0fa8ed8e-a456-4902-a178-79a0ac87d67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:18.290372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:53:18.307456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1366/2733] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:53:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:53:18.376486 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f34a15-80f4-4c95-acba-06e3efeb0836 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:18.376955 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f34a15-80f4-4c95-acba-06e3efeb0836 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:18.377272 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f34a15-80f4-4c95-acba-06e3efeb0836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:18.414791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-45f34a15-80f4-4c95-acba-06e3efeb0836 None None] MFA Rules not processed for user `a70f695bbb7b494ab85f253ad5285e5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:53:18.417232 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-45f34a15-80f4-4c95-acba-06e3efeb0836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:18.497836 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1368/2734] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:53:18 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:53:18.512923 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8ed8e-a456-4902-a178-79a0ac87d67c None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:18.514359 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-0fa8ed8e-a456-4902-a178-79a0ac87d67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:18.629890 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f34a15-80f4-4c95-acba-06e3efeb0836 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:18.631095 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8ed8e-a456-4902-a178-79a0ac87d67c None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:18.637759 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b2e8f-331e-4187-9913-20099e98ca4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:18.638551 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b2e8f-331e-4187-9913-20099e98ca4f service nova] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:18.638740 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b2e8f-331e-4187-9913-20099e98ca4f service nova] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:18.639013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b2e8f-331e-4187-9913-20099e98ca4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:18.691213 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f34a15-80f4-4c95-acba-06e3efeb0836 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:18.695033 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d485720-e131-4374-9597-e1f72bef896f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:18.695791 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d485720-e131-4374-9597-e1f72bef896f service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:18.696138 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d485720-e131-4374-9597-e1f72bef896f service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:18.696480 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d485720-e131-4374-9597-e1f72bef896f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:18.699153 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-7d485720-e131-4374-9597-e1f72bef896f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:18.707202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b2e8f-331e-4187-9913-20099e98ca4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:18.710009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b2e8f-331e-4187-9913-20099e98ca4f service nova] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:18.710801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d485720-e131-4374-9597-e1f72bef896f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:18.713851 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d485720-e131-4374-9597-e1f72bef896f service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:18.723974 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1367/2735] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:53:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:53:18.731902 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1369/2736] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:53:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:53:19.906205 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b2e8f-331e-4187-9913-20099e98ca4f service nova] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:19.907832 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-7a2b2e8f-331e-4187-9913-20099e98ca4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:20.008584 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b2e8f-331e-4187-9913-20099e98ca4f service nova] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:20.014764 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c70fc9-aac6-4c34-8619-b9278a1f0730 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cd6f0a1286243e4add692db01028205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:20.015405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c70fc9-aac6-4c34-8619-b9278a1f0730 service placement] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:20.015687 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c70fc9-aac6-4c34-8619-b9278a1f0730 service placement] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:20.015959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c70fc9-aac6-4c34-8619-b9278a1f0730 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:20.017709 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-42c70fc9-aac6-4c34-8619-b9278a1f0730 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:20.041299 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c70fc9-aac6-4c34-8619-b9278a1f0730 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:20.043892 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c70fc9-aac6-4c34-8619-b9278a1f0730 service placement] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:20.054896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1368/2737] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:53:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:53:22.257466 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b9427-adb6-4c6f-8bde-0bef660ed400 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:22.257993 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b9427-adb6-4c6f-8bde-0bef660ed400 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:22.258118 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b9427-adb6-4c6f-8bde-0bef660ed400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:22.285200 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-342b9427-adb6-4c6f-8bde-0bef660ed400 None None] MFA Rules not processed for user `58e1f1ac6ea14da0be6e9a24d007847a`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:53:22.286512 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-342b9427-adb6-4c6f-8bde-0bef660ed400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:22.340797 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:53:22.361564 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1370/2738] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Jun 27 18:53:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:53:22.379153 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c70fc9-aac6-4c34-8619-b9278a1f0730 service placement] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:22.411927 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=100 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:53:22.492532 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c70fc9-aac6-4c34-8619-b9278a1f0730 service placement] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:22.502173 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e9c03-c0ac-4359-b093-96056bb3acb8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1529c9a45434a19968e924797688f06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:22.503013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e9c03-c0ac-4359-b093-96056bb3acb8 service swift] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:22.503593 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e9c03-c0ac-4359-b093-96056bb3acb8 service swift] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:22.504004 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e9c03-c0ac-4359-b093-96056bb3acb8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:22.513877 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e9c03-c0ac-4359-b093-96056bb3acb8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:22.515711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e9c03-c0ac-4359-b093-96056bb3acb8 service swift] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:22.525790 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1369/2739] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Jun 27 18:53:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 18:53:32.522423 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=22 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:53:32.542705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=5 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:53:38.496947 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20a3bc-0947-4020-ab7f-c4379a2a7757 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:38.497579 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20a3bc-0947-4020-ab7f-c4379a2a7757 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:38.497579 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20a3bc-0947-4020-ab7f-c4379a2a7757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:38.515411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-7a20a3bc-0947-4020-ab7f-c4379a2a7757 None None] MFA Rules not processed for user `b10a90cce1b048d7bd55938811301ab4`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:53:38.516649 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-7a20a3bc-0947-4020-ab7f-c4379a2a7757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:38.523431 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1371/2740] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:53:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:53:43.562278 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29aee3-4e7f-4814-8f28-3b410add64dc None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:43.562800 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29aee3-4e7f-4814-8f28-3b410add64dc None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:43.562900 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29aee3-4e7f-4814-8f28-3b410add64dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:43.583807 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-eb29aee3-4e7f-4814-8f28-3b410add64dc None None] MFA Rules not processed for user `48ac761d30de4fb49478a78ccf23ab92`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:53:43.585009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-eb29aee3-4e7f-4814-8f28-3b410add64dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:43.659328 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1370/2741] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:53:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:53:46.431059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20a3bc-0947-4020-ab7f-c4379a2a7757 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:46.441751 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20a3bc-0947-4020-ab7f-c4379a2a7757 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:46.445331 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ab4f9-c125-4794-9ea4-49aa64d40b00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:46.445970 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ab4f9-c125-4794-9ea4-49aa64d40b00 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:46.446252 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ab4f9-c125-4794-9ea4-49aa64d40b00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:46.446558 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ab4f9-c125-4794-9ea4-49aa64d40b00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:46.448349 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-e24ab4f9-c125-4794-9ea4-49aa64d40b00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:46.512034 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ab4f9-c125-4794-9ea4-49aa64d40b00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:46.514361 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ab4f9-c125-4794-9ea4-49aa64d40b00 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:46.523430 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1372/2742] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:53:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:53:52.531292 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29aee3-4e7f-4814-8f28-3b410add64dc None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:52.532701 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-eb29aee3-4e7f-4814-8f28-3b410add64dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:52.545565 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29aee3-4e7f-4814-8f28-3b410add64dc None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:52.549650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c07fb-22e9-4eb0-873e-2efe5d1f55a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:52.553458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c07fb-22e9-4eb0-873e-2efe5d1f55a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:52.553458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c07fb-22e9-4eb0-873e-2efe5d1f55a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:52.553458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c07fb-22e9-4eb0-873e-2efe5d1f55a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:52.618527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c07fb-22e9-4eb0-873e-2efe5d1f55a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:52.620048 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c07fb-22e9-4eb0-873e-2efe5d1f55a7 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:52.639146 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1371/2743] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:53:52 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:53:54.830691 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ab4f9-c125-4794-9ea4-49aa64d40b00 service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:54.841142 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ab4f9-c125-4794-9ea4-49aa64d40b00 service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:54.845714 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e5542-c622-4aaf-b07c-2789c1c7c911 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:54.846532 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e5542-c622-4aaf-b07c-2789c1c7c911 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:54.847067 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e5542-c622-4aaf-b07c-2789c1c7c911 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:54.847563 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e5542-c622-4aaf-b07c-2789c1c7c911 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:54.849519 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-248e5542-c622-4aaf-b07c-2789c1c7c911 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:53:54.976732 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e5542-c622-4aaf-b07c-2789c1c7c911 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:54.978902 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e5542-c622-4aaf-b07c-2789c1c7c911 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:54.994729 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1373/2744] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:53:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:53:59.877020 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c07fb-22e9-4eb0-873e-2efe5d1f55a7 service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:53:59.893236 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c07fb-22e9-4eb0-873e-2efe5d1f55a7 service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:53:59.897834 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf863e11-9ff2-42a2-9593-8c7b7d5e0b58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:53:59.899501 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf863e11-9ff2-42a2-9593-8c7b7d5e0b58 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:53:59.899983 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf863e11-9ff2-42a2-9593-8c7b7d5e0b58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:53:59.900430 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf863e11-9ff2-42a2-9593-8c7b7d5e0b58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:53:59.931568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf863e11-9ff2-42a2-9593-8c7b7d5e0b58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:53:59.933043 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf863e11-9ff2-42a2-9593-8c7b7d5e0b58 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:53:59.956527 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1372/2745] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:53:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:54:04.408043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e5542-c622-4aaf-b07c-2789c1c7c911 service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:54:04.409944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-248e5542-c622-4aaf-b07c-2789c1c7c911 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:54:04.418938 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=62 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:54:04.433313 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e5542-c622-4aaf-b07c-2789c1c7c911 service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:54:04.447081 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e8d12-5d49-4ff9-92f6-6714820860b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:54:04.448008 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e8d12-5d49-4ff9-92f6-6714820860b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:54:04.448298 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e8d12-5d49-4ff9-92f6-6714820860b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:54:04.448570 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e8d12-5d49-4ff9-92f6-6714820860b1 admin admin] PATH_INFO: `/v3/users/49ccbcfbd48642aabd74d789b1875dd3` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:54:04.449905 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e8d12-5d49-4ff9-92f6-6714820860b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49ccbcfbd48642aabd74d789b1875dd3)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:54:04.451960 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e8d12-5d49-4ff9-92f6-6714820860b1 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:54:04.676579 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2c0e8d12-5d49-4ff9-92f6-6714820860b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49ccbcfbd48642aabd74d789b1875dd3'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:54:04.683868 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2c0e8d12-5d49-4ff9-92f6-6714820860b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49ccbcfbd48642aabd74d789b1875dd3'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:54:04.684577 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2c0e8d12-5d49-4ff9-92f6-6714820860b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49ccbcfbd48642aabd74d789b1875dd3'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:54:04.698004 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2c0e8d12-5d49-4ff9-92f6-6714820860b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49ccbcfbd48642aabd74d789b1875dd3'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:54:04.764002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2c0e8d12-5d49-4ff9-92f6-6714820860b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49ccbcfbd48642aabd74d789b1875dd3'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:54:04.779010 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1374/2746] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:54:04 2022] DELETE /identity/v3/users/49ccbcfbd48642aabd74d789b1875dd3 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:54:05.030872 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf863e11-9ff2-42a2-9593-8c7b7d5e0b58 service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:54:05.042152 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf863e11-9ff2-42a2-9593-8c7b7d5e0b58 service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:54:05.106513 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42367651-21e4-4d1b-9295-12c3e191fec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:54:05.108364 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42367651-21e4-4d1b-9295-12c3e191fec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:54:05.108867 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42367651-21e4-4d1b-9295-12c3e191fec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:54:05.109164 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42367651-21e4-4d1b-9295-12c3e191fec1 admin admin] PATH_INFO: `/v3/projects/61866ab089f04e9abfab3f6ce58cf6b8` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:54:05.110375 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42367651-21e4-4d1b-9295-12c3e191fec1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61866ab089f04e9abfab3f6ce58cf6b8)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:54:05.112327 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42367651-21e4-4d1b-9295-12c3e191fec1 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:54:05.200813 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:54:05.215893 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-42367651-21e4-4d1b-9295-12c3e191fec1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61866ab089f04e9abfab3f6ce58cf6b8', 'request_id': 'req-42367651-21e4-4d1b-9295-12c3e191fec1'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:54:05.329805 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-42367651-21e4-4d1b-9295-12c3e191fec1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61866ab089f04e9abfab3f6ce58cf6b8', 'request_id': 'req-42367651-21e4-4d1b-9295-12c3e191fec1'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:54:05.337297 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-42367651-21e4-4d1b-9295-12c3e191fec1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61866ab089f04e9abfab3f6ce58cf6b8', 'request_id': 'req-42367651-21e4-4d1b-9295-12c3e191fec1'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:54:05.349552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-42367651-21e4-4d1b-9295-12c3e191fec1 admin admin] The token cache is being invalidate because project 61866ab089f04e9abfab3f6ce58cf6b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:54:05.349840 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-42367651-21e4-4d1b-9295-12c3e191fec1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:54:05.350523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-42367651-21e4-4d1b-9295-12c3e191fec1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:54:05.352074 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1373/2747] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:54:05 2022] DELETE /identity/v3/projects/61866ab089f04e9abfab3f6ce58cf6b8 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:54:05.356069 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e8d12-5d49-4ff9-92f6-6714820860b1 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:54:05.357609 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-2c0e8d12-5d49-4ff9-92f6-6714820860b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:54:05.371100 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e8d12-5d49-4ff9-92f6-6714820860b1 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:54:05.427562 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6a121-3b21-4d23-a459-84eb2521c87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:54:05.428457 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6a121-3b21-4d23-a459-84eb2521c87d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:54:05.428736 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6a121-3b21-4d23-a459-84eb2521c87d admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:54:05.429011 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6a121-3b21-4d23-a459-84eb2521c87d admin admin] PATH_INFO: `/v3/users/b4a8f980f48a4de2a3db6cfbf718beab` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:54:05.430184 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6a121-3b21-4d23-a459-84eb2521c87d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4a8f980f48a4de2a3db6cfbf718beab)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:54:05.432196 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6a121-3b21-4d23-a459-84eb2521c87d admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:54:05.637312 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-9cf6a121-3b21-4d23-a459-84eb2521c87d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4a8f980f48a4de2a3db6cfbf718beab'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:54:05.645534 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-9cf6a121-3b21-4d23-a459-84eb2521c87d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4a8f980f48a4de2a3db6cfbf718beab'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:54:05.646219 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-9cf6a121-3b21-4d23-a459-84eb2521c87d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4a8f980f48a4de2a3db6cfbf718beab'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:54:05.658812 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-9cf6a121-3b21-4d23-a459-84eb2521c87d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4a8f980f48a4de2a3db6cfbf718beab'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:54:05.661538 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:54:05.748116 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-9cf6a121-3b21-4d23-a459-84eb2521c87d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4a8f980f48a4de2a3db6cfbf718beab'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:54:05.756003 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1375/2748] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:54:05 2022] DELETE /identity/v3/users/b4a8f980f48a4de2a3db6cfbf718beab => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:54:06.001844 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42367651-21e4-4d1b-9295-12c3e191fec1 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:54:06.003372 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-42367651-21e4-4d1b-9295-12c3e191fec1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:54:06.017472 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42367651-21e4-4d1b-9295-12c3e191fec1 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:54:06.075737 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59991e10-2594-45dc-b2fe-8ecfa8a2ba7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:54:06.076566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59991e10-2594-45dc-b2fe-8ecfa8a2ba7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:54:06.076851 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59991e10-2594-45dc-b2fe-8ecfa8a2ba7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:54:06.077130 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59991e10-2594-45dc-b2fe-8ecfa8a2ba7f admin admin] PATH_INFO: `/v3/projects/48eeb3fe7e194bf9aea90d74a07ec266` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:54:06.078296 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59991e10-2594-45dc-b2fe-8ecfa8a2ba7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=48eeb3fe7e194bf9aea90d74a07ec266)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:54:06.080263 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59991e10-2594-45dc-b2fe-8ecfa8a2ba7f admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:54:06.200568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-59991e10-2594-45dc-b2fe-8ecfa8a2ba7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48eeb3fe7e194bf9aea90d74a07ec266', 'request_id': 'req-59991e10-2594-45dc-b2fe-8ecfa8a2ba7f'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:54:06.277434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-59991e10-2594-45dc-b2fe-8ecfa8a2ba7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48eeb3fe7e194bf9aea90d74a07ec266', 'request_id': 'req-59991e10-2594-45dc-b2fe-8ecfa8a2ba7f'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:54:06.283866 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-59991e10-2594-45dc-b2fe-8ecfa8a2ba7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48eeb3fe7e194bf9aea90d74a07ec266', 'request_id': 'req-59991e10-2594-45dc-b2fe-8ecfa8a2ba7f'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:54:06.294298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-59991e10-2594-45dc-b2fe-8ecfa8a2ba7f admin admin] The token cache is being invalidate because project 48eeb3fe7e194bf9aea90d74a07ec266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:54:06.294616 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-59991e10-2594-45dc-b2fe-8ecfa8a2ba7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:54:06.296271 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-59991e10-2594-45dc-b2fe-8ecfa8a2ba7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:54:06.297815 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1374/2749] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:54:06 2022] DELETE /identity/v3/projects/48eeb3fe7e194bf9aea90d74a07ec266 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:54:15.805371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:DELETE=1,keystone:SELECT=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:54:16.291464 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=65,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:54:53.610628 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb47e2-f2ee-4fe0-be2f-87d4bfee7b11 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:54:53.612974 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb47e2-f2ee-4fe0-be2f-87d4bfee7b11 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:54:53.612974 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb47e2-f2ee-4fe0-be2f-87d4bfee7b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:54:53.628572 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-feeb47e2-f2ee-4fe0-be2f-87d4bfee7b11 None None] MFA Rules not processed for user `48ac761d30de4fb49478a78ccf23ab92`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:54:53.629778 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-feeb47e2-f2ee-4fe0-be2f-87d4bfee7b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:54:53.683947 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1376/2750] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:54:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:55:03.677361 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=25 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:55:04.821143 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59991e10-2594-45dc-b2fe-8ecfa8a2ba7f admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:55:04.822654 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-59991e10-2594-45dc-b2fe-8ecfa8a2ba7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:55:04.879360 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59991e10-2594-45dc-b2fe-8ecfa8a2ba7f admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:55:04.882911 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a459d7d-8fdb-49fa-a8ff-2873d1d52764 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:55:04.883544 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a459d7d-8fdb-49fa-a8ff-2873d1d52764 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:55:04.883858 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a459d7d-8fdb-49fa-a8ff-2873d1d52764 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:55:04.884140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a459d7d-8fdb-49fa-a8ff-2873d1d52764 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:55:04.885924 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-1a459d7d-8fdb-49fa-a8ff-2873d1d52764 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:55:04.958680 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a459d7d-8fdb-49fa-a8ff-2873d1d52764 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:55:04.960126 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a459d7d-8fdb-49fa-a8ff-2873d1d52764 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:55:04.972244 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1375/2751] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:55:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:55:06.143778 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb47e2-f2ee-4fe0-be2f-87d4bfee7b11 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:55:06.153828 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb47e2-f2ee-4fe0-be2f-87d4bfee7b11 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:55:06.157373 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e22c39-50f6-4776-821f-6083a4d686d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:55:06.158034 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e22c39-50f6-4776-821f-6083a4d686d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:55:06.158314 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e22c39-50f6-4776-821f-6083a4d686d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:55:06.158620 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e22c39-50f6-4776-821f-6083a4d686d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:55:06.160373 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-13e22c39-50f6-4776-821f-6083a4d686d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:55:06.174072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e22c39-50f6-4776-821f-6083a4d686d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:55:06.175513 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e22c39-50f6-4776-821f-6083a4d686d0 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:55:06.186906 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1377/2752] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:55:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:55:14.969275 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=54 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:55:16.181766 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=6 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:55:21.297484 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e446dc2-5c30-4827-9717-22976ea5f74e None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:55:21.297976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e446dc2-5c30-4827-9717-22976ea5f74e None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:55:21.298104 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e446dc2-5c30-4827-9717-22976ea5f74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:55:21.315231 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-8e446dc2-5c30-4827-9717-22976ea5f74e None None] MFA Rules not processed for user `58e1f1ac6ea14da0be6e9a24d007847a`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:55:21.316411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-8e446dc2-5c30-4827-9717-22976ea5f74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:55:21.371876 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1376/2753] 10.222.0.15 () {58 vars in 967 bytes} [Mon Jun 27 18:55:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:55:21.412961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e22c39-50f6-4776-821f-6083a4d686d0 service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:55:21.414435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-13e22c39-50f6-4776-821f-6083a4d686d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:55:21.483149 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e22c39-50f6-4776-821f-6083a4d686d0 service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:55:21.488133 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d8a94-e2d1-48d3-9056-05215996002e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1529c9a45434a19968e924797688f06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:55:21.488785 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d8a94-e2d1-48d3-9056-05215996002e service swift] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:55:21.489068 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d8a94-e2d1-48d3-9056-05215996002e service swift] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:55:21.489344 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d8a94-e2d1-48d3-9056-05215996002e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:55:21.497366 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d8a94-e2d1-48d3-9056-05215996002e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:55:21.499213 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d8a94-e2d1-48d3-9056-05215996002e service swift] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:55:21.507047 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1378/2754] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Jun 27 18:55:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 18:55:25.851079 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e446dc2-5c30-4827-9717-22976ea5f74e None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:55:25.852870 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-8e446dc2-5c30-4827-9717-22976ea5f74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:55:25.926479 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e446dc2-5c30-4827-9717-22976ea5f74e None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:55:25.931610 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a600e80-4c0e-4ddc-9daf-973a60cc67eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cd6f0a1286243e4add692db01028205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:55:25.932631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a600e80-4c0e-4ddc-9daf-973a60cc67eb service placement] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:55:25.932914 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a600e80-4c0e-4ddc-9daf-973a60cc67eb service placement] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:55:25.933217 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a600e80-4c0e-4ddc-9daf-973a60cc67eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:55:25.935048 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-3a600e80-4c0e-4ddc-9daf-973a60cc67eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:55:25.948045 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a600e80-4c0e-4ddc-9daf-973a60cc67eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:55:25.949707 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a600e80-4c0e-4ddc-9daf-973a60cc67eb service placement] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:55:25.960141 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1377/2755] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:55:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:55:29.533106 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d8a94-e2d1-48d3-9056-05215996002e service swift] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:55:29.534567 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-697d8a94-e2d1-48d3-9056-05215996002e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:55:29.602027 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d8a94-e2d1-48d3-9056-05215996002e service swift] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:55:29.606588 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e88a5-433f-4d7b-a050-b76b5a0a0deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:55:29.607475 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e88a5-433f-4d7b-a050-b76b5a0a0deb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:55:29.607759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e88a5-433f-4d7b-a050-b76b5a0a0deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:55:29.608034 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e88a5-433f-4d7b-a050-b76b5a0a0deb admin admin] PATH_INFO: `/v3/users/5d58cb1968f240d693e4c414869ffe44` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:55:29.609236 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e88a5-433f-4d7b-a050-b76b5a0a0deb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d58cb1968f240d693e4c414869ffe44)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:55:29.611183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e88a5-433f-4d7b-a050-b76b5a0a0deb admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:55:29.771395 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c19e88a5-433f-4d7b-a050-b76b5a0a0deb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d58cb1968f240d693e4c414869ffe44'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:55:29.779441 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c19e88a5-433f-4d7b-a050-b76b5a0a0deb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d58cb1968f240d693e4c414869ffe44'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:55:29.780119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c19e88a5-433f-4d7b-a050-b76b5a0a0deb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d58cb1968f240d693e4c414869ffe44'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:55:29.791969 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c19e88a5-433f-4d7b-a050-b76b5a0a0deb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d58cb1968f240d693e4c414869ffe44'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:55:29.854116 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c19e88a5-433f-4d7b-a050-b76b5a0a0deb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d58cb1968f240d693e4c414869ffe44'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:55:29.860271 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1379/2756] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:55:29 2022] DELETE /identity/v3/users/5d58cb1968f240d693e4c414869ffe44 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:55:30.101657 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a600e80-4c0e-4ddc-9daf-973a60cc67eb service placement] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:55:30.113581 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a600e80-4c0e-4ddc-9daf-973a60cc67eb service placement] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:55:30.175035 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c680a3-e511-4201-9dca-9304975f3251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:55:30.175886 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c680a3-e511-4201-9dca-9304975f3251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:55:30.176166 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c680a3-e511-4201-9dca-9304975f3251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:55:30.176435 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c680a3-e511-4201-9dca-9304975f3251 admin admin] PATH_INFO: `/v3/projects/22ff7f833172448fa47fbb796cd91c3d` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:55:30.177619 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c680a3-e511-4201-9dca-9304975f3251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22ff7f833172448fa47fbb796cd91c3d)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:55:30.179566 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c680a3-e511-4201-9dca-9304975f3251 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:55:30.268858 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:55:30.277318 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b4c680a3-e511-4201-9dca-9304975f3251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22ff7f833172448fa47fbb796cd91c3d', 'request_id': 'req-b4c680a3-e511-4201-9dca-9304975f3251'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:55:30.351344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b4c680a3-e511-4201-9dca-9304975f3251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22ff7f833172448fa47fbb796cd91c3d', 'request_id': 'req-b4c680a3-e511-4201-9dca-9304975f3251'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:55:30.358649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b4c680a3-e511-4201-9dca-9304975f3251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22ff7f833172448fa47fbb796cd91c3d', 'request_id': 'req-b4c680a3-e511-4201-9dca-9304975f3251'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:55:30.369975 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b4c680a3-e511-4201-9dca-9304975f3251 admin admin] The token cache is being invalidate because project 22ff7f833172448fa47fbb796cd91c3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:55:30.370259 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b4c680a3-e511-4201-9dca-9304975f3251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:55:30.370904 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b4c680a3-e511-4201-9dca-9304975f3251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:55:30.372420 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1378/2757] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:55:30 2022] DELETE /identity/v3/projects/22ff7f833172448fa47fbb796cd91c3d => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:55:30.376599 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e88a5-433f-4d7b-a050-b76b5a0a0deb admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:55:30.377991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-c19e88a5-433f-4d7b-a050-b76b5a0a0deb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:55:30.390932 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e88a5-433f-4d7b-a050-b76b5a0a0deb admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:55:30.395458 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:55:30.447925 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0bdb4c-0bc8-41c9-9fd0-dad83e93c87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:55:30.448782 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0bdb4c-0bc8-41c9-9fd0-dad83e93c87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:55:30.449061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0bdb4c-0bc8-41c9-9fd0-dad83e93c87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:55:30.449336 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0bdb4c-0bc8-41c9-9fd0-dad83e93c87e admin admin] PATH_INFO: `/v3/users/b3e359535c0546d1897d9addb5fae985` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:55:30.450592 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0bdb4c-0bc8-41c9-9fd0-dad83e93c87e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3e359535c0546d1897d9addb5fae985)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:55:30.452538 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0bdb4c-0bc8-41c9-9fd0-dad83e93c87e admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:55:30.602990 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-eb0bdb4c-0bc8-41c9-9fd0-dad83e93c87e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3e359535c0546d1897d9addb5fae985'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:55:30.609804 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-eb0bdb4c-0bc8-41c9-9fd0-dad83e93c87e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3e359535c0546d1897d9addb5fae985'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:55:30.611370 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-eb0bdb4c-0bc8-41c9-9fd0-dad83e93c87e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3e359535c0546d1897d9addb5fae985'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:55:30.623350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-eb0bdb4c-0bc8-41c9-9fd0-dad83e93c87e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3e359535c0546d1897d9addb5fae985'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:55:30.693813 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-eb0bdb4c-0bc8-41c9-9fd0-dad83e93c87e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3e359535c0546d1897d9addb5fae985'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:55:30.700032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1380/2758] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:55:30 2022] DELETE /identity/v3/users/b3e359535c0546d1897d9addb5fae985 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:55:30.981310 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c680a3-e511-4201-9dca-9304975f3251 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:55:30.982994 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-b4c680a3-e511-4201-9dca-9304975f3251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:55:30.996724 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c680a3-e511-4201-9dca-9304975f3251 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:55:31.055277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e96394e-e9b9-4351-bbd0-1dbb7c8a03de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:55:31.056180 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e96394e-e9b9-4351-bbd0-1dbb7c8a03de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:55:31.056465 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e96394e-e9b9-4351-bbd0-1dbb7c8a03de admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:55:31.056742 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e96394e-e9b9-4351-bbd0-1dbb7c8a03de admin admin] PATH_INFO: `/v3/projects/7e62b539f1fa447794a1ff02384d28cd` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:55:31.057962 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e96394e-e9b9-4351-bbd0-1dbb7c8a03de admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e62b539f1fa447794a1ff02384d28cd)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:55:31.059982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e96394e-e9b9-4351-bbd0-1dbb7c8a03de admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:55:31.166360 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-0e96394e-e9b9-4351-bbd0-1dbb7c8a03de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e62b539f1fa447794a1ff02384d28cd', 'request_id': 'req-0e96394e-e9b9-4351-bbd0-1dbb7c8a03de'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:55:31.234541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-0e96394e-e9b9-4351-bbd0-1dbb7c8a03de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e62b539f1fa447794a1ff02384d28cd', 'request_id': 'req-0e96394e-e9b9-4351-bbd0-1dbb7c8a03de'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:55:31.240999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-0e96394e-e9b9-4351-bbd0-1dbb7c8a03de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e62b539f1fa447794a1ff02384d28cd', 'request_id': 'req-0e96394e-e9b9-4351-bbd0-1dbb7c8a03de'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:55:31.251574 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-0e96394e-e9b9-4351-bbd0-1dbb7c8a03de admin admin] The token cache is being invalidate because project 7e62b539f1fa447794a1ff02384d28cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:55:31.251854 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-0e96394e-e9b9-4351-bbd0-1dbb7c8a03de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:55:31.252503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-0e96394e-e9b9-4351-bbd0-1dbb7c8a03de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:55:31.255568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1379/2759] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:55:30 2022] DELETE /identity/v3/projects/7e62b539f1fa447794a1ff02384d28cd => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:55:40.696265 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:55:41.248673 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=62,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:55:58.080189 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0bdb4c-0bc8-41c9-9fd0-dad83e93c87e admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:55:58.081651 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-eb0bdb4c-0bc8-41c9-9fd0-dad83e93c87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:55:58.175999 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0bdb4c-0bc8-41c9-9fd0-dad83e93c87e admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:55:58.254581 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a6104-d156-4bd0-9238-27a5e1ffbf7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:55:58.255359 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a6104-d156-4bd0-9238-27a5e1ffbf7a service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:55:58.255737 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a6104-d156-4bd0-9238-27a5e1ffbf7a service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:55:58.256072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a6104-d156-4bd0-9238-27a5e1ffbf7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:55:58.258150 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-1c9a6104-d156-4bd0-9238-27a5e1ffbf7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:55:58.275117 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a6104-d156-4bd0-9238-27a5e1ffbf7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:55:58.277703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a6104-d156-4bd0-9238-27a5e1ffbf7a service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:55:58.291041 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1381/2760] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:55:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:56:02.253765 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4b148-d31c-4024-a1fc-7ebf49617cd2 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:56:02.254271 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4b148-d31c-4024-a1fc-7ebf49617cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:56:02.254371 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4b148-d31c-4024-a1fc-7ebf49617cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:56:02.272719 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-99c4b148-d31c-4024-a1fc-7ebf49617cd2 None None] MFA Rules not processed for user `48ac761d30de4fb49478a78ccf23ab92`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:56:02.273969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-99c4b148-d31c-4024-a1fc-7ebf49617cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:56:02.329165 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1380/2761] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:56:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:56:03.104851 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a6104-d156-4bd0-9238-27a5e1ffbf7a service nova] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:56:03.107120 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-1c9a6104-d156-4bd0-9238-27a5e1ffbf7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:56:03.185852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a6104-d156-4bd0-9238-27a5e1ffbf7a service nova] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:56:03.190816 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766c809-b72d-4831-85fe-e86d7c188356 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cd6f0a1286243e4add692db01028205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:56:03.191457 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766c809-b72d-4831-85fe-e86d7c188356 service placement] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:56:03.191736 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766c809-b72d-4831-85fe-e86d7c188356 service placement] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:56:03.192005 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766c809-b72d-4831-85fe-e86d7c188356 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:56:03.193752 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-7766c809-b72d-4831-85fe-e86d7c188356 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:56:03.205595 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766c809-b72d-4831-85fe-e86d7c188356 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:56:03.207211 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766c809-b72d-4831-85fe-e86d7c188356 service placement] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:56:03.217744 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1382/2762] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:56:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:56:06.202438 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643406a-a370-46f7-ae6d-d554726344fb None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:56:06.203292 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643406a-a370-46f7-ae6d-d554726344fb None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:56:06.203573 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643406a-a370-46f7-ae6d-d554726344fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:56:06.222075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-b643406a-a370-46f7-ae6d-d554726344fb None None] MFA Rules not processed for user `b10a90cce1b048d7bd55938811301ab4`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:56:06.223419 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-b643406a-a370-46f7-ae6d-d554726344fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:56:06.241922 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766c809-b72d-4831-85fe-e86d7c188356 service placement] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:56:06.245467 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-7766c809-b72d-4831-85fe-e86d7c188356 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:56:06.260991 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=100 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:56:06.293866 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1381/2763] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:56:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:56:06.309104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766c809-b72d-4831-85fe-e86d7c188356 service placement] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:56:06.313018 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a0cfa-1f27-41b8-acec-180103a6dbc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:56:06.313669 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a0cfa-1f27-41b8-acec-180103a6dbc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:56:06.313954 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a0cfa-1f27-41b8-acec-180103a6dbc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:56:06.314234 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a0cfa-1f27-41b8-acec-180103a6dbc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:56:06.316130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-134a0cfa-1f27-41b8-acec-180103a6dbc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:56:06.329830 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a0cfa-1f27-41b8-acec-180103a6dbc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:56:06.331312 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a0cfa-1f27-41b8-acec-180103a6dbc8 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:56:06.340537 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1383/2764] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:56:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:56:12.011960 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643406a-a370-46f7-ae6d-d554726344fb None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:56:12.013475 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-b643406a-a370-46f7-ae6d-d554726344fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:56:12.031778 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643406a-a370-46f7-ae6d-d554726344fb None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:56:12.037589 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee5127-96fb-4b5f-8d67-ff8043a499b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:56:12.038446 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee5127-96fb-4b5f-8d67-ff8043a499b6 service nova] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:56:12.038803 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee5127-96fb-4b5f-8d67-ff8043a499b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:56:12.039145 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee5127-96fb-4b5f-8d67-ff8043a499b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:56:12.041255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-2aee5127-96fb-4b5f-8d67-ff8043a499b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:56:12.054892 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee5127-96fb-4b5f-8d67-ff8043a499b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:56:12.057534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee5127-96fb-4b5f-8d67-ff8043a499b6 service nova] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:56:12.080064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1382/2765] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:56:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:56:16.277655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a0cfa-1f27-41b8-acec-180103a6dbc8 service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:56:16.290775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a0cfa-1f27-41b8-acec-180103a6dbc8 service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:56:16.297172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bbc110-c468-4cc4-9244-005f684532ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dddd4efb44c4152bb4151f7715f2223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:56:16.297881 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbc110-c468-4cc4-9244-005f684532ce service nova] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:56:16.298158 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbc110-c468-4cc4-9244-005f684532ce service nova] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:56:16.298467 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbc110-c468-4cc4-9244-005f684532ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:56:16.300265 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-51bbc110-c468-4cc4-9244-005f684532ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:56:16.312707 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bbc110-c468-4cc4-9244-005f684532ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:56:16.314288 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bbc110-c468-4cc4-9244-005f684532ce service nova] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:56:16.325200 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1384/2766] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:56:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:56:22.064601 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=56 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:56:26.320442 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=26 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:12.417058 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee5127-96fb-4b5f-8d67-ff8043a499b6 service nova] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:12.419878 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-2aee5127-96fb-4b5f-8d67-ff8043a499b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:12.548018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee5127-96fb-4b5f-8d67-ff8043a499b6 service nova] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:12.552672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c28953-42c7-436d-bad5-0a26dc49e887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:12.553535 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c28953-42c7-436d-bad5-0a26dc49e887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:12.553819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c28953-42c7-436d-bad5-0a26dc49e887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:12.554098 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c28953-42c7-436d-bad5-0a26dc49e887 admin admin] PATH_INFO: `/v3/users/48ac761d30de4fb49478a78ccf23ab92` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:12.555327 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c28953-42c7-436d-bad5-0a26dc49e887 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48ac761d30de4fb49478a78ccf23ab92)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:12.557335 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c28953-42c7-436d-bad5-0a26dc49e887 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:12.756921 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-d7c28953-42c7-436d-bad5-0a26dc49e887 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48ac761d30de4fb49478a78ccf23ab92'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:12.763695 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-d7c28953-42c7-436d-bad5-0a26dc49e887 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48ac761d30de4fb49478a78ccf23ab92'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:12.764385 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-d7c28953-42c7-436d-bad5-0a26dc49e887 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48ac761d30de4fb49478a78ccf23ab92'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:12.764989 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-d7c28953-42c7-436d-bad5-0a26dc49e887 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48ac761d30de4fb49478a78ccf23ab92'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:12.776822 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-d7c28953-42c7-436d-bad5-0a26dc49e887 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48ac761d30de4fb49478a78ccf23ab92'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:12.873489 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1383/2767] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:57:12 2022] DELETE /identity/v3/users/48ac761d30de4fb49478a78ccf23ab92 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:13.104649 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bbc110-c468-4cc4-9244-005f684532ce service nova] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:13.142503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bbc110-c468-4cc4-9244-005f684532ce service nova] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:13.199021 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a389b-befc-4cd1-bd3f-d4fdaad95808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:13.199866 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a389b-befc-4cd1-bd3f-d4fdaad95808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:13.200144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a389b-befc-4cd1-bd3f-d4fdaad95808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:13.200417 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a389b-befc-4cd1-bd3f-d4fdaad95808 admin admin] PATH_INFO: `/v3/projects/a981ae734b1e4993bb440918b1a74fe7` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:13.201576 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a389b-befc-4cd1-bd3f-d4fdaad95808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a981ae734b1e4993bb440918b1a74fe7)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:13.204093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a389b-befc-4cd1-bd3f-d4fdaad95808 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:13.338901 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c42a389b-befc-4cd1-bd3f-d4fdaad95808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a981ae734b1e4993bb440918b1a74fe7', 'request_id': 'req-c42a389b-befc-4cd1-bd3f-d4fdaad95808'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:13.400020 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c42a389b-befc-4cd1-bd3f-d4fdaad95808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a981ae734b1e4993bb440918b1a74fe7', 'request_id': 'req-c42a389b-befc-4cd1-bd3f-d4fdaad95808'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:13.407101 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c42a389b-befc-4cd1-bd3f-d4fdaad95808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a981ae734b1e4993bb440918b1a74fe7', 'request_id': 'req-c42a389b-befc-4cd1-bd3f-d4fdaad95808'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:13.417828 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c42a389b-befc-4cd1-bd3f-d4fdaad95808 admin admin] The token cache is being invalidate because project a981ae734b1e4993bb440918b1a74fe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:57:13.418117 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c42a389b-befc-4cd1-bd3f-d4fdaad95808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:13.418770 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-c42a389b-befc-4cd1-bd3f-d4fdaad95808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:13.420304 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1385/2768] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:57:13 2022] DELETE /identity/v3/projects/a981ae734b1e4993bb440918b1a74fe7 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:57:13.425952 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ca295-fcab-4033-9f03-2a89f879bf6b None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:13.426523 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ca295-fcab-4033-9f03-2a89f879bf6b None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:13.426626 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ca295-fcab-4033-9f03-2a89f879bf6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:13.452782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:13.473059 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-147ca295-fcab-4033-9f03-2a89f879bf6b None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:13.474243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-147ca295-fcab-4033-9f03-2a89f879bf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:13.532913 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1384/2769] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:13.540775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce418d-2a32-400a-96ea-2e6f9ec4d800 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:13.541412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce418d-2a32-400a-96ea-2e6f9ec4d800 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:13.541689 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce418d-2a32-400a-96ea-2e6f9ec4d800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:13.560010 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-dfce418d-2a32-400a-96ea-2e6f9ec4d800 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:13.561190 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-dfce418d-2a32-400a-96ea-2e6f9ec4d800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:13.569051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1386/2770] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:13.573287 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ca295-fcab-4033-9f03-2a89f879bf6b None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:13.583476 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ca295-fcab-4033-9f03-2a89f879bf6b None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:13.588297 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13742ff7-36c2-4fdc-9473-f74cbb3a1c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:13.588976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13742ff7-36c2-4fdc-9473-f74cbb3a1c50 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:13.589262 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13742ff7-36c2-4fdc-9473-f74cbb3a1c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:13.589539 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13742ff7-36c2-4fdc-9473-f74cbb3a1c50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:13.590266 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13742ff7-36c2-4fdc-9473-f74cbb3a1c50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:57:13.590608 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13742ff7-36c2-4fdc-9473-f74cbb3a1c50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:13.592230 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13742ff7-36c2-4fdc-9473-f74cbb3a1c50 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:13.604427 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1385/2771] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:57:13.608621 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce418d-2a32-400a-96ea-2e6f9ec4d800 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:13.617775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce418d-2a32-400a-96ea-2e6f9ec4d800 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:13.622401 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c966876-ce1f-4b9b-b0ba-2c35b5f00b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:13.623086 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c966876-ce1f-4b9b-b0ba-2c35b5f00b12 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:13.623365 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c966876-ce1f-4b9b-b0ba-2c35b5f00b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:13.623637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c966876-ce1f-4b9b-b0ba-2c35b5f00b12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:13.624327 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c966876-ce1f-4b9b-b0ba-2c35b5f00b12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:13.626674 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c966876-ce1f-4b9b-b0ba-2c35b5f00b12 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:13.744034 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1387/2772] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:57:13 2022] POST /identity/v3/projects => generated 365 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:57:13.745447 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13742ff7-36c2-4fdc-9473-f74cbb3a1c50 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:13.756744 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13742ff7-36c2-4fdc-9473-f74cbb3a1c50 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:13.811366 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c6e8e-1e1c-4c31-969a-37d3fb3be79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:13.812022 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c6e8e-1e1c-4c31-969a-37d3fb3be79f admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:13.812925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c6e8e-1e1c-4c31-969a-37d3fb3be79f admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:13.813203 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c6e8e-1e1c-4c31-969a-37d3fb3be79f admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:13.813893 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c6e8e-1e1c-4c31-969a-37d3fb3be79f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:13.815875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c6e8e-1e1c-4c31-969a-37d3fb3be79f admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:13.926327 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1386/2773] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:57:13 2022] POST /identity/v3/users => generated 283 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:57:13.930492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c966876-ce1f-4b9b-b0ba-2c35b5f00b12 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:13.941380 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c966876-ce1f-4b9b-b0ba-2c35b5f00b12 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:13.946036 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e69cc0-497b-42ae-9904-52140acc0d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:13.946722 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e69cc0-497b-42ae-9904-52140acc0d3d admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:13.947002 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e69cc0-497b-42ae-9904-52140acc0d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:13.947275 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e69cc0-497b-42ae-9904-52140acc0d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:13.947998 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e69cc0-497b-42ae-9904-52140acc0d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:13.949580 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e69cc0-497b-42ae-9904-52140acc0d3d admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:13.960133 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1388/2774] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:57:13.966841 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c6e8e-1e1c-4c31-969a-37d3fb3be79f admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:13.977715 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c6e8e-1e1c-4c31-969a-37d3fb3be79f admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:13.982362 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e95671-56e7-4815-a01c-11f73b7aa804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:13.983046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e95671-56e7-4815-a01c-11f73b7aa804 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:13.983333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e95671-56e7-4815-a01c-11f73b7aa804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:13.983614 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e95671-56e7-4815-a01c-11f73b7aa804 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:13.985055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e95671-56e7-4815-a01c-11f73b7aa804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:13.986695 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e95671-56e7-4815-a01c-11f73b7aa804 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:13.996529 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1387/2775] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:57:14.001218 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e69cc0-497b-42ae-9904-52140acc0d3d admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:14.011006 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e69cc0-497b-42ae-9904-52140acc0d3d admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:14.016232 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff65a4-0526-4abc-a657-7d268ece5fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:14.017092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff65a4-0526-4abc-a657-7d268ece5fc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:14.017371 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff65a4-0526-4abc-a657-7d268ece5fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:14.017645 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff65a4-0526-4abc-a657-7d268ece5fc1 admin admin] PATH_INFO: `/v3/projects/436c2a0914144b36a4d64dcd16f39dff/users/9ce91830df61444e83db714909bf14d7/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:14.033123 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff65a4-0526-4abc-a657-7d268ece5fc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=436c2a0914144b36a4d64dcd16f39dff, user_id=9ce91830df61444e83db714909bf14d7, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:14.035786 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff65a4-0526-4abc-a657-7d268ece5fc1 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:14.142708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1389/2776] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:57:13 2022] PUT /identity/v3/projects/436c2a0914144b36a4d64dcd16f39dff/users/9ce91830df61444e83db714909bf14d7/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:14.153431 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5b3b2-7555-447f-9e4c-ce8af88aa0b8 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:14.153730 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5b3b2-7555-447f-9e4c-ce8af88aa0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:14.154013 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca5b3b2-7555-447f-9e4c-ce8af88aa0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:14.181050 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-eca5b3b2-7555-447f-9e4c-ce8af88aa0b8 None None] MFA Rules not processed for user `9ce91830df61444e83db714909bf14d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:14.182258 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-eca5b3b2-7555-447f-9e4c-ce8af88aa0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:14.228598 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:14.245684 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1388/2777] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:14.250624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff65a4-0526-4abc-a657-7d268ece5fc1 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:14.255059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:14.312803 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff65a4-0526-4abc-a657-7d268ece5fc1 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:14.317562 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282764aa-b291-44af-8aeb-b3932ba16daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:14.318212 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282764aa-b291-44af-8aeb-b3932ba16daf admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:14.318529 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282764aa-b291-44af-8aeb-b3932ba16daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:14.318809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282764aa-b291-44af-8aeb-b3932ba16daf admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:14.319503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282764aa-b291-44af-8aeb-b3932ba16daf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:14.321429 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282764aa-b291-44af-8aeb-b3932ba16daf admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:14.402823 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1390/2778] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:57:14 2022] POST /identity/v3/projects => generated 365 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:57:14.407510 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5b3b2-7555-447f-9e4c-ce8af88aa0b8 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:14.418243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca5b3b2-7555-447f-9e4c-ce8af88aa0b8 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:14.475370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffbef1-5293-42e2-aea6-b1530725c93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:14.476073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffbef1-5293-42e2-aea6-b1530725c93b admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:14.476362 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffbef1-5293-42e2-aea6-b1530725c93b admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:14.476642 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffbef1-5293-42e2-aea6-b1530725c93b admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:14.477325 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ffbef1-5293-42e2-aea6-b1530725c93b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:14.479275 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ffbef1-5293-42e2-aea6-b1530725c93b admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:14.590180 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1389/2779] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:57:14 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:57:14.594763 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282764aa-b291-44af-8aeb-b3932ba16daf admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:14.604954 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282764aa-b291-44af-8aeb-b3932ba16daf admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:14.609524 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a949ab0-cbdd-42d1-8993-6fc6814133fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:14.610167 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a949ab0-cbdd-42d1-8993-6fc6814133fe admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:14.610479 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a949ab0-cbdd-42d1-8993-6fc6814133fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:14.610754 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a949ab0-cbdd-42d1-8993-6fc6814133fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:14.611468 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a949ab0-cbdd-42d1-8993-6fc6814133fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:14.613037 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a949ab0-cbdd-42d1-8993-6fc6814133fe admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:14.623733 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1391/2780] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:57:14.628411 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffbef1-5293-42e2-aea6-b1530725c93b admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:14.638146 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffbef1-5293-42e2-aea6-b1530725c93b admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:14.643458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c923b-7243-46f0-a09b-8a690cb20ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:14.644334 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c923b-7243-46f0-a09b-8a690cb20ee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:14.644622 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c923b-7243-46f0-a09b-8a690cb20ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:14.644898 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c923b-7243-46f0-a09b-8a690cb20ee8 admin admin] PATH_INFO: `/v3/projects/879c478d532847fabc5cad8add22e7ba/users/e1808c13026e4be2879400222dc643fd/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:14.660328 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c923b-7243-46f0-a09b-8a690cb20ee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=879c478d532847fabc5cad8add22e7ba, user_id=e1808c13026e4be2879400222dc643fd, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:14.663016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c923b-7243-46f0-a09b-8a690cb20ee8 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:14.739277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1390/2781] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:57:14 2022] PUT /identity/v3/projects/879c478d532847fabc5cad8add22e7ba/users/e1808c13026e4be2879400222dc643fd/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:14.745104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a949ab0-cbdd-42d1-8993-6fc6814133fe admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:14.755921 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a949ab0-cbdd-42d1-8993-6fc6814133fe admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:14.811806 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dda609-04f6-4d93-80cd-958defbdf278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:14.813090 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dda609-04f6-4d93-80cd-958defbdf278 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:14.813377 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dda609-04f6-4d93-80cd-958defbdf278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:14.813657 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dda609-04f6-4d93-80cd-958defbdf278 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:14.814408 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dda609-04f6-4d93-80cd-958defbdf278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:14.816009 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dda609-04f6-4d93-80cd-958defbdf278 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:14.825856 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1392/2782] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:57:14.830100 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c923b-7243-46f0-a09b-8a690cb20ee8 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:14.840534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c923b-7243-46f0-a09b-8a690cb20ee8 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:14.845760 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7814c-1126-4e40-b3b2-0f90f8cc90a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:14.846795 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7814c-1126-4e40-b3b2-0f90f8cc90a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:14.847081 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7814c-1126-4e40-b3b2-0f90f8cc90a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:14.847354 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7814c-1126-4e40-b3b2-0f90f8cc90a1 admin admin] PATH_INFO: `/v3/projects/879c478d532847fabc5cad8add22e7ba/users/e1808c13026e4be2879400222dc643fd/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:14.849323 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7814c-1126-4e40-b3b2-0f90f8cc90a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=879c478d532847fabc5cad8add22e7ba, user_id=e1808c13026e4be2879400222dc643fd, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:14.852019 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7814c-1126-4e40-b3b2-0f90f8cc90a1 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:14.863055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1391/2783] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:57:14 2022] PUT /identity/v3/projects/879c478d532847fabc5cad8add22e7ba/users/e1808c13026e4be2879400222dc643fd/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:14.872894 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc94686-bd1f-4315-96b6-bbd08b503f9a None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:14.873184 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc94686-bd1f-4315-96b6-bbd08b503f9a None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:14.873464 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc94686-bd1f-4315-96b6-bbd08b503f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:14.900836 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-abc94686-bd1f-4315-96b6-bbd08b503f9a None None] MFA Rules not processed for user `e1808c13026e4be2879400222dc643fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:14.902042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-abc94686-bd1f-4315-96b6-bbd08b503f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:14.963437 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:14.968869 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1393/2784] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:14 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:14.980264 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7814c-1126-4e40-b3b2-0f90f8cc90a1 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:14.982020 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-e5c7814c-1126-4e40-b3b2-0f90f8cc90a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:15.056835 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7814c-1126-4e40-b3b2-0f90f8cc90a1 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:15.060575 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41859077-9561-4fe8-a55a-a032c95591bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:15.061212 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41859077-9561-4fe8-a55a-a032c95591bb service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:15.061493 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41859077-9561-4fe8-a55a-a032c95591bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:15.061772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41859077-9561-4fe8-a55a-a032c95591bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:15.071077 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41859077-9561-4fe8-a55a-a032c95591bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:15.072501 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41859077-9561-4fe8-a55a-a032c95591bb service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:15.082434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1392/2785] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:57:14 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:57:15.294779 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc94686-bd1f-4315-96b6-bbd08b503f9a None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:15.304624 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc94686-bd1f-4315-96b6-bbd08b503f9a None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:15.308257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e906aad-d53d-4c01-b4d6-dcaba80758c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:15.308881 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e906aad-d53d-4c01-b4d6-dcaba80758c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:15.309158 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e906aad-d53d-4c01-b4d6-dcaba80758c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:15.309428 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e906aad-d53d-4c01-b4d6-dcaba80758c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:15.366646 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e906aad-d53d-4c01-b4d6-dcaba80758c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:15.368104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e906aad-d53d-4c01-b4d6-dcaba80758c2 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:15.377678 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1394/2786] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:57:15 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:57:18.873367 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41859077-9561-4fe8-a55a-a032c95591bb service cinder] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:18.906490 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:18.934777 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41859077-9561-4fe8-a55a-a032c95591bb service cinder] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:18.940117 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbaaad-6195-4fce-9e77-44540089700a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:18.940975 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbaaad-6195-4fce-9e77-44540089700a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:18.941261 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbaaad-6195-4fce-9e77-44540089700a admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:18.941536 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbaaad-6195-4fce-9e77-44540089700a admin admin] PATH_INFO: `/v3/users/9ce91830df61444e83db714909bf14d7` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:18.942744 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cbaaad-6195-4fce-9e77-44540089700a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ce91830df61444e83db714909bf14d7)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:18.944739 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cbaaad-6195-4fce-9e77-44540089700a admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:19.095274 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-39cbaaad-6195-4fce-9e77-44540089700a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ce91830df61444e83db714909bf14d7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:19.102525 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-39cbaaad-6195-4fce-9e77-44540089700a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ce91830df61444e83db714909bf14d7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:19.103234 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-39cbaaad-6195-4fce-9e77-44540089700a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ce91830df61444e83db714909bf14d7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:19.103872 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-39cbaaad-6195-4fce-9e77-44540089700a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ce91830df61444e83db714909bf14d7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:19.116631 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-39cbaaad-6195-4fce-9e77-44540089700a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ce91830df61444e83db714909bf14d7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:19.184239 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1393/2787] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:57:18 2022] DELETE /identity/v3/users/9ce91830df61444e83db714909bf14d7 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:19.188864 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e04cf-935b-49a7-9b83-9bd54a2e2f3a None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:19.189367 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e04cf-935b-49a7-9b83-9bd54a2e2f3a None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:19.189367 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e04cf-935b-49a7-9b83-9bd54a2e2f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:19.206048 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-da5e04cf-935b-49a7-9b83-9bd54a2e2f3a None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:19.207315 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-da5e04cf-935b-49a7-9b83-9bd54a2e2f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:19.265859 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1395/2788] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:19.276234 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbaaad-6195-4fce-9e77-44540089700a admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:19.277160 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-39cbaaad-6195-4fce-9e77-44540089700a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:19.354035 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbaaad-6195-4fce-9e77-44540089700a admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:19.357730 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9e926-7dbb-4859-b42d-16aac7fe46e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:19.358364 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9e926-7dbb-4859-b42d-16aac7fe46e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:19.358677 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9e926-7dbb-4859-b42d-16aac7fe46e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:19.358952 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9e926-7dbb-4859-b42d-16aac7fe46e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:19.372167 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9e926-7dbb-4859-b42d-16aac7fe46e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:19.372167 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9e926-7dbb-4859-b42d-16aac7fe46e5 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:19.380006 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1394/2789] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:57:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:57:19.769393 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e04cf-935b-49a7-9b83-9bd54a2e2f3a None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:19.770880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-da5e04cf-935b-49a7-9b83-9bd54a2e2f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:19.783662 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e04cf-935b-49a7-9b83-9bd54a2e2f3a None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:19.788264 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35f025-cf41-49bf-895c-8620d116e52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:19.789125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35f025-cf41-49bf-895c-8620d116e52a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:19.789431 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35f025-cf41-49bf-895c-8620d116e52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:19.789677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff35f025-cf41-49bf-895c-8620d116e52a admin admin] PATH_INFO: `/v3/projects/436c2a0914144b36a4d64dcd16f39dff` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:19.790892 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35f025-cf41-49bf-895c-8620d116e52a admin admin] RBAC: Authorizing `identity:delete_project(project_id=436c2a0914144b36a4d64dcd16f39dff)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:19.792805 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff35f025-cf41-49bf-895c-8620d116e52a admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:19.882812 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-ff35f025-cf41-49bf-895c-8620d116e52a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '436c2a0914144b36a4d64dcd16f39dff', 'request_id': 'req-ff35f025-cf41-49bf-895c-8620d116e52a'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:19.955915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-ff35f025-cf41-49bf-895c-8620d116e52a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '436c2a0914144b36a4d64dcd16f39dff', 'request_id': 'req-ff35f025-cf41-49bf-895c-8620d116e52a'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:19.962705 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-ff35f025-cf41-49bf-895c-8620d116e52a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '436c2a0914144b36a4d64dcd16f39dff', 'request_id': 'req-ff35f025-cf41-49bf-895c-8620d116e52a'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:19.973729 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-ff35f025-cf41-49bf-895c-8620d116e52a admin admin] The token cache is being invalidate because project 436c2a0914144b36a4d64dcd16f39dff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:57:19.974018 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-ff35f025-cf41-49bf-895c-8620d116e52a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:19.974685 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-ff35f025-cf41-49bf-895c-8620d116e52a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:19.976428 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1396/2790] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:57:19 2022] DELETE /identity/v3/projects/436c2a0914144b36a4d64dcd16f39dff => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:19.980294 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9e926-7dbb-4859-b42d-16aac7fe46e5 service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:19.990677 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9e926-7dbb-4859-b42d-16aac7fe46e5 service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:20.020585 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:20.048525 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33e5d1-d049-4f84-a3f9-4cc70851017b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:20.049710 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33e5d1-d049-4f84-a3f9-4cc70851017b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:20.050044 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33e5d1-d049-4f84-a3f9-4cc70851017b admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:20.050321 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33e5d1-d049-4f84-a3f9-4cc70851017b admin admin] PATH_INFO: `/v3/users/e1808c13026e4be2879400222dc643fd` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:20.051547 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33e5d1-d049-4f84-a3f9-4cc70851017b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1808c13026e4be2879400222dc643fd)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:20.053550 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33e5d1-d049-4f84-a3f9-4cc70851017b admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:20.197891 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-1a33e5d1-d049-4f84-a3f9-4cc70851017b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1808c13026e4be2879400222dc643fd'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:20.204821 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-1a33e5d1-d049-4f84-a3f9-4cc70851017b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1808c13026e4be2879400222dc643fd'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:20.205504 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-1a33e5d1-d049-4f84-a3f9-4cc70851017b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1808c13026e4be2879400222dc643fd'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:20.206140 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-1a33e5d1-d049-4f84-a3f9-4cc70851017b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1808c13026e4be2879400222dc643fd'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:20.217999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-1a33e5d1-d049-4f84-a3f9-4cc70851017b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1808c13026e4be2879400222dc643fd'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:20.281052 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1395/2791] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:57:19 2022] DELETE /identity/v3/users/e1808c13026e4be2879400222dc643fd => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:20.700068 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35f025-cf41-49bf-895c-8620d116e52a admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:20.701583 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-ff35f025-cf41-49bf-895c-8620d116e52a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:20.715393 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff35f025-cf41-49bf-895c-8620d116e52a admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:20.732973 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:20.776430 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43441bca-edb5-4745-a9d8-8ad54644bafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:20.777307 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43441bca-edb5-4745-a9d8-8ad54644bafd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:20.777584 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43441bca-edb5-4745-a9d8-8ad54644bafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:20.777858 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43441bca-edb5-4745-a9d8-8ad54644bafd admin admin] PATH_INFO: `/v3/projects/879c478d532847fabc5cad8add22e7ba` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:20.779043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43441bca-edb5-4745-a9d8-8ad54644bafd admin admin] RBAC: Authorizing `identity:delete_project(project_id=879c478d532847fabc5cad8add22e7ba)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:20.780952 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43441bca-edb5-4745-a9d8-8ad54644bafd admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:20.886828 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-43441bca-edb5-4745-a9d8-8ad54644bafd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '879c478d532847fabc5cad8add22e7ba', 'request_id': 'req-43441bca-edb5-4745-a9d8-8ad54644bafd'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:20.942299 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-43441bca-edb5-4745-a9d8-8ad54644bafd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '879c478d532847fabc5cad8add22e7ba', 'request_id': 'req-43441bca-edb5-4745-a9d8-8ad54644bafd'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:20.949277 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-43441bca-edb5-4745-a9d8-8ad54644bafd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '879c478d532847fabc5cad8add22e7ba', 'request_id': 'req-43441bca-edb5-4745-a9d8-8ad54644bafd'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:20.960462 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-43441bca-edb5-4745-a9d8-8ad54644bafd admin admin] The token cache is being invalidate because project 879c478d532847fabc5cad8add22e7ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:57:20.960686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-43441bca-edb5-4745-a9d8-8ad54644bafd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:20.961334 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-43441bca-edb5-4745-a9d8-8ad54644bafd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:20.962880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1397/2792] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:57:20 2022] DELETE /identity/v3/projects/879c478d532847fabc5cad8add22e7ba => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:20.968059 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a5e1d-8a54-4c6d-bea9-46f27a81f194 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:20.968354 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a5e1d-8a54-4c6d-bea9-46f27a81f194 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:20.968743 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a5e1d-8a54-4c6d-bea9-46f27a81f194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:20.986568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-c14a5e1d-8a54-4c6d-bea9-46f27a81f194 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:20.987803 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c14a5e1d-8a54-4c6d-bea9-46f27a81f194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:21.046302 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1396/2793] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:21.056356 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ef2ef-3860-443f-beea-f4a6684c5793 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:21.056698 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ef2ef-3860-443f-beea-f4a6684c5793 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:21.056973 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ef2ef-3860-443f-beea-f4a6684c5793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:21.075967 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-e84ef2ef-3860-443f-beea-f4a6684c5793 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:21.079043 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-e84ef2ef-3860-443f-beea-f4a6684c5793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:21.085552 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1398/2794] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:21.089811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a5e1d-8a54-4c6d-bea9-46f27a81f194 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:21.099650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a5e1d-8a54-4c6d-bea9-46f27a81f194 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:21.104158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cf473-5532-4fdf-b8a0-20b63f3193ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:21.104793 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cf473-5532-4fdf-b8a0-20b63f3193ed admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:21.105071 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cf473-5532-4fdf-b8a0-20b63f3193ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:21.105341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cf473-5532-4fdf-b8a0-20b63f3193ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:21.106056 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56cf473-5532-4fdf-b8a0-20b63f3193ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:57:21.106363 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56cf473-5532-4fdf-b8a0-20b63f3193ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:21.107963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56cf473-5532-4fdf-b8a0-20b63f3193ed admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:21.120029 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1397/2795] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:57:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:57:21.124410 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ef2ef-3860-443f-beea-f4a6684c5793 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:21.134285 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ef2ef-3860-443f-beea-f4a6684c5793 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:21.138959 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f9829-c9aa-4a0a-a8a1-44b4a1879395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:21.139595 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f9829-c9aa-4a0a-a8a1-44b4a1879395 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:21.139877 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f9829-c9aa-4a0a-a8a1-44b4a1879395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:21.140152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f9829-c9aa-4a0a-a8a1-44b4a1879395 admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:21.140968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f9829-c9aa-4a0a-a8a1-44b4a1879395 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:21.142977 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f9829-c9aa-4a0a-a8a1-44b4a1879395 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:21.238355 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1399/2796] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:57:21 2022] POST /identity/v3/projects => generated 369 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:57:21.242225 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cf473-5532-4fdf-b8a0-20b63f3193ed admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:21.253074 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cf473-5532-4fdf-b8a0-20b63f3193ed admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:21.290815 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:21.309158 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c3354-db3d-4463-9553-e2039dd5faf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:21.309819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c3354-db3d-4463-9553-e2039dd5faf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:21.310103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c3354-db3d-4463-9553-e2039dd5faf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:21.310393 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c3354-db3d-4463-9553-e2039dd5faf1 admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:21.311111 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c3354-db3d-4463-9553-e2039dd5faf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:21.313656 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c3354-db3d-4463-9553-e2039dd5faf1 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:21.408189 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1398/2797] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:57:21 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:57:21.412339 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f9829-c9aa-4a0a-a8a1-44b4a1879395 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:21.423824 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f9829-c9aa-4a0a-a8a1-44b4a1879395 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:21.428406 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04596f-15a3-47fe-811b-8b7f5a06bf5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:21.429063 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04596f-15a3-47fe-811b-8b7f5a06bf5b admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:21.429343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04596f-15a3-47fe-811b-8b7f5a06bf5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:21.429616 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04596f-15a3-47fe-811b-8b7f5a06bf5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:21.430332 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04596f-15a3-47fe-811b-8b7f5a06bf5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:21.431955 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04596f-15a3-47fe-811b-8b7f5a06bf5b admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:21.442749 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1400/2798] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:57:21.446702 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c3354-db3d-4463-9553-e2039dd5faf1 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:21.457054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c3354-db3d-4463-9553-e2039dd5faf1 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:21.461816 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8212f-6136-482d-819a-1638cfb059aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:21.462617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8212f-6136-482d-819a-1638cfb059aa admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:21.462904 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8212f-6136-482d-819a-1638cfb059aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:21.463183 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8212f-6136-482d-819a-1638cfb059aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:21.463929 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8212f-6136-482d-819a-1638cfb059aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:21.465524 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8212f-6136-482d-819a-1638cfb059aa admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:21.476883 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1399/2799] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:57:21.481101 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04596f-15a3-47fe-811b-8b7f5a06bf5b admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:21.491971 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04596f-15a3-47fe-811b-8b7f5a06bf5b admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:21.496570 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008ce17-0986-4a1d-9bd9-337cee258eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:21.497427 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008ce17-0986-4a1d-9bd9-337cee258eac admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:21.497710 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008ce17-0986-4a1d-9bd9-337cee258eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:21.498001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008ce17-0986-4a1d-9bd9-337cee258eac admin admin] PATH_INFO: `/v3/projects/508e844283e749caa508f6a9ab39710c/users/372274c4b96f410886c75173bcf90bb2/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:21.513352 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008ce17-0986-4a1d-9bd9-337cee258eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=508e844283e749caa508f6a9ab39710c, user_id=372274c4b96f410886c75173bcf90bb2, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:21.516163 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008ce17-0986-4a1d-9bd9-337cee258eac admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:21.596435 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1401/2800] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:57:21 2022] PUT /identity/v3/projects/508e844283e749caa508f6a9ab39710c/users/372274c4b96f410886c75173bcf90bb2/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:21.606650 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2a737-6ec2-461a-9912-65f8f069b1c4 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:21.606940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2a737-6ec2-461a-9912-65f8f069b1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:21.607211 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2a737-6ec2-461a-9912-65f8f069b1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:21.634097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-dfe2a737-6ec2-461a-9912-65f8f069b1c4 None None] MFA Rules not processed for user `372274c4b96f410886c75173bcf90bb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:21.635329 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-dfe2a737-6ec2-461a-9912-65f8f069b1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:21.700827 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1400/2801] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:21 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:21.705724 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008ce17-0986-4a1d-9bd9-337cee258eac admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:21.757953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:21.772968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008ce17-0986-4a1d-9bd9-337cee258eac admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:21.777917 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82735eb4-b512-48de-8fbf-85e1b6e1e54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:21.778674 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82735eb4-b512-48de-8fbf-85e1b6e1e54c admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:21.780935 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82735eb4-b512-48de-8fbf-85e1b6e1e54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:21.780935 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82735eb4-b512-48de-8fbf-85e1b6e1e54c admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:21.780935 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82735eb4-b512-48de-8fbf-85e1b6e1e54c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:21.783053 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82735eb4-b512-48de-8fbf-85e1b6e1e54c admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:21.865201 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1402/2802] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:57:21 2022] POST /identity/v3/projects => generated 365 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:57:21.876754 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2a737-6ec2-461a-9912-65f8f069b1c4 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:21.887519 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2a737-6ec2-461a-9912-65f8f069b1c4 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:21.949062 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade26871-55d1-4b64-a088-300a120f3ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:21.949727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade26871-55d1-4b64-a088-300a120f3ece admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:21.950040 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade26871-55d1-4b64-a088-300a120f3ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:21.950271 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade26871-55d1-4b64-a088-300a120f3ece admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:21.951006 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade26871-55d1-4b64-a088-300a120f3ece admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:21.953604 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade26871-55d1-4b64-a088-300a120f3ece admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:22.050540 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1401/2803] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:57:21 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:57:22.051684 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82735eb4-b512-48de-8fbf-85e1b6e1e54c admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:22.062224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82735eb4-b512-48de-8fbf-85e1b6e1e54c admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:22.067277 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd22249-eb06-4605-ae03-e03f162d869a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:22.067939 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd22249-eb06-4605-ae03-e03f162d869a admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:22.068225 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd22249-eb06-4605-ae03-e03f162d869a admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:22.068552 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd22249-eb06-4605-ae03-e03f162d869a admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:22.069316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd22249-eb06-4605-ae03-e03f162d869a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:22.071022 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd22249-eb06-4605-ae03-e03f162d869a admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:22.082616 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1403/2804] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:57:22.088748 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade26871-55d1-4b64-a088-300a120f3ece admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:22.100182 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade26871-55d1-4b64-a088-300a120f3ece admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:22.105118 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9c12a-ae90-40b8-a3bc-ee7609fc565b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:22.106255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9c12a-ae90-40b8-a3bc-ee7609fc565b admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:22.106255 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9c12a-ae90-40b8-a3bc-ee7609fc565b admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:22.107552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9c12a-ae90-40b8-a3bc-ee7609fc565b admin admin] PATH_INFO: `/v3/projects/b5662afebfe64aeab2973b5b6ec7a7be/users/144025f3e2aa498f9b6eb4787e860159/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:22.123586 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa9c12a-ae90-40b8-a3bc-ee7609fc565b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5662afebfe64aeab2973b5b6ec7a7be, user_id=144025f3e2aa498f9b6eb4787e860159, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:22.127214 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa9c12a-ae90-40b8-a3bc-ee7609fc565b admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:22.208568 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1402/2805] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:57:22 2022] PUT /identity/v3/projects/b5662afebfe64aeab2973b5b6ec7a7be/users/144025f3e2aa498f9b6eb4787e860159/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:22.212490 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd22249-eb06-4605-ae03-e03f162d869a admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:22.222905 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd22249-eb06-4605-ae03-e03f162d869a admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:22.280880 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0f744-82c1-4023-967a-0a628cb5778b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:22.281574 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0f744-82c1-4023-967a-0a628cb5778b admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:22.281856 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0f744-82c1-4023-967a-0a628cb5778b admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:22.282129 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0f744-82c1-4023-967a-0a628cb5778b admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:22.282944 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0f744-82c1-4023-967a-0a628cb5778b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:22.284570 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0f744-82c1-4023-967a-0a628cb5778b admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:22.294694 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1404/2806] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:57:22.299779 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9c12a-ae90-40b8-a3bc-ee7609fc565b admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:22.311289 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9c12a-ae90-40b8-a3bc-ee7609fc565b admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:22.318977 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed79c4-fd08-49ca-ab8f-2ce87050546d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:22.318977 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed79c4-fd08-49ca-ab8f-2ce87050546d admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:22.318977 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed79c4-fd08-49ca-ab8f-2ce87050546d admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:22.318977 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed79c4-fd08-49ca-ab8f-2ce87050546d admin admin] PATH_INFO: `/v3/projects/b5662afebfe64aeab2973b5b6ec7a7be/users/144025f3e2aa498f9b6eb4787e860159/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:22.320389 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed79c4-fd08-49ca-ab8f-2ce87050546d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5662afebfe64aeab2973b5b6ec7a7be, user_id=144025f3e2aa498f9b6eb4787e860159, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:22.323104 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed79c4-fd08-49ca-ab8f-2ce87050546d admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:22.330297 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:22.334927 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1403/2807] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:57:22 2022] PUT /identity/v3/projects/b5662afebfe64aeab2973b5b6ec7a7be/users/144025f3e2aa498f9b6eb4787e860159/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:22.345030 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccfd46-6848-4d8c-b930-eed0e164eb80 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:22.345317 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccfd46-6848-4d8c-b930-eed0e164eb80 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:22.345596 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccfd46-6848-4d8c-b930-eed0e164eb80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:22.371702 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-eaccfd46-6848-4d8c-b930-eed0e164eb80 None None] MFA Rules not processed for user `144025f3e2aa498f9b6eb4787e860159`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:22.373011 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-eaccfd46-6848-4d8c-b930-eed0e164eb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:22.440937 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1405/2808] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:22 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:22.451963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed79c4-fd08-49ca-ab8f-2ce87050546d admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:22.454493 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c2ed79c4-fd08-49ca-ab8f-2ce87050546d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:22.519333 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed79c4-fd08-49ca-ab8f-2ce87050546d admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:22.523080 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62605b30-3997-45c0-85e0-9ec6d458286f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:22.523770 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62605b30-3997-45c0-85e0-9ec6d458286f service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:22.524054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62605b30-3997-45c0-85e0-9ec6d458286f service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:22.524327 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62605b30-3997-45c0-85e0-9ec6d458286f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:22.533393 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62605b30-3997-45c0-85e0-9ec6d458286f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:22.534801 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62605b30-3997-45c0-85e0-9ec6d458286f service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:22.544440 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1404/2809] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:57:22 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:57:25.268307 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccfd46-6848-4d8c-b930-eed0e164eb80 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:25.311424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:25.329424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccfd46-6848-4d8c-b930-eed0e164eb80 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:25.333946 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454081a-ebc1-435f-9137-aeda4d00c4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:25.334758 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454081a-ebc1-435f-9137-aeda4d00c4eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:25.335035 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454081a-ebc1-435f-9137-aeda4d00c4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:25.335304 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454081a-ebc1-435f-9137-aeda4d00c4eb admin admin] PATH_INFO: `/v3/users/372274c4b96f410886c75173bcf90bb2` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:25.336476 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454081a-ebc1-435f-9137-aeda4d00c4eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=372274c4b96f410886c75173bcf90bb2)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:25.338373 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454081a-ebc1-435f-9137-aeda4d00c4eb admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:25.539872 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-7454081a-ebc1-435f-9137-aeda4d00c4eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '372274c4b96f410886c75173bcf90bb2'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:25.546848 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-7454081a-ebc1-435f-9137-aeda4d00c4eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '372274c4b96f410886c75173bcf90bb2'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:25.548313 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-7454081a-ebc1-435f-9137-aeda4d00c4eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '372274c4b96f410886c75173bcf90bb2'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:25.560255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-7454081a-ebc1-435f-9137-aeda4d00c4eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '372274c4b96f410886c75173bcf90bb2'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:25.628232 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-7454081a-ebc1-435f-9137-aeda4d00c4eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '372274c4b96f410886c75173bcf90bb2'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:25.634649 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1406/2810] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:57:25 2022] DELETE /identity/v3/users/372274c4b96f410886c75173bcf90bb2 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:25.639436 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8749f82-f6e3-4519-822e-f52e0d795ae8 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:25.639928 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8749f82-f6e3-4519-822e-f52e0d795ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:25.640030 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8749f82-f6e3-4519-822e-f52e0d795ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:25.658705 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-a8749f82-f6e3-4519-822e-f52e0d795ae8 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:25.659942 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-a8749f82-f6e3-4519-822e-f52e0d795ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:25.720204 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1405/2811] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:25.729740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454081a-ebc1-435f-9137-aeda4d00c4eb admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:25.731773 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-7454081a-ebc1-435f-9137-aeda4d00c4eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:25.791910 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454081a-ebc1-435f-9137-aeda4d00c4eb admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:25.795531 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65f3f7-27a0-4ba2-a362-cc4e9d043094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:25.796166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65f3f7-27a0-4ba2-a362-cc4e9d043094 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:25.796446 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65f3f7-27a0-4ba2-a362-cc4e9d043094 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:25.796719 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65f3f7-27a0-4ba2-a362-cc4e9d043094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:25.806494 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65f3f7-27a0-4ba2-a362-cc4e9d043094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:25.807900 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65f3f7-27a0-4ba2-a362-cc4e9d043094 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:25.818274 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1407/2812] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:57:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:57:26.248547 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8749f82-f6e3-4519-822e-f52e0d795ae8 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:26.249974 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-a8749f82-f6e3-4519-822e-f52e0d795ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:26.264575 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8749f82-f6e3-4519-822e-f52e0d795ae8 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:26.269086 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802dd8ae-c878-4fc1-a00d-3c207e15eaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:26.269986 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802dd8ae-c878-4fc1-a00d-3c207e15eaa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:26.270301 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802dd8ae-c878-4fc1-a00d-3c207e15eaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:26.270653 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802dd8ae-c878-4fc1-a00d-3c207e15eaa7 admin admin] PATH_INFO: `/v3/projects/508e844283e749caa508f6a9ab39710c` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:26.271873 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802dd8ae-c878-4fc1-a00d-3c207e15eaa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=508e844283e749caa508f6a9ab39710c)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:26.273795 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802dd8ae-c878-4fc1-a00d-3c207e15eaa7 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:26.385606 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-802dd8ae-c878-4fc1-a00d-3c207e15eaa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '508e844283e749caa508f6a9ab39710c', 'request_id': 'req-802dd8ae-c878-4fc1-a00d-3c207e15eaa7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:26.469940 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-802dd8ae-c878-4fc1-a00d-3c207e15eaa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '508e844283e749caa508f6a9ab39710c', 'request_id': 'req-802dd8ae-c878-4fc1-a00d-3c207e15eaa7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:26.477118 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-802dd8ae-c878-4fc1-a00d-3c207e15eaa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '508e844283e749caa508f6a9ab39710c', 'request_id': 'req-802dd8ae-c878-4fc1-a00d-3c207e15eaa7'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:26.488534 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-802dd8ae-c878-4fc1-a00d-3c207e15eaa7 admin admin] The token cache is being invalidate because project 508e844283e749caa508f6a9ab39710c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:57:26.488826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-802dd8ae-c878-4fc1-a00d-3c207e15eaa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:26.489467 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-802dd8ae-c878-4fc1-a00d-3c207e15eaa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:26.491094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1406/2813] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:57:26 2022] DELETE /identity/v3/projects/508e844283e749caa508f6a9ab39710c => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:26.495327 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65f3f7-27a0-4ba2-a362-cc4e9d043094 service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:26.505909 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65f3f7-27a0-4ba2-a362-cc4e9d043094 service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:26.561694 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3cb82f-a848-44fe-8569-8c084df440f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:26.562589 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3cb82f-a848-44fe-8569-8c084df440f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:26.562874 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3cb82f-a848-44fe-8569-8c084df440f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:26.563152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3cb82f-a848-44fe-8569-8c084df440f1 admin admin] PATH_INFO: `/v3/users/144025f3e2aa498f9b6eb4787e860159` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:26.565237 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3cb82f-a848-44fe-8569-8c084df440f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=144025f3e2aa498f9b6eb4787e860159)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:26.567190 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3cb82f-a848-44fe-8569-8c084df440f1 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:26.571694 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:26.741759 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-1c3cb82f-a848-44fe-8569-8c084df440f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '144025f3e2aa498f9b6eb4787e860159'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:26.752906 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-1c3cb82f-a848-44fe-8569-8c084df440f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '144025f3e2aa498f9b6eb4787e860159'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:26.753597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-1c3cb82f-a848-44fe-8569-8c084df440f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '144025f3e2aa498f9b6eb4787e860159'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:26.766299 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-1c3cb82f-a848-44fe-8569-8c084df440f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '144025f3e2aa498f9b6eb4787e860159'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:26.841064 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-1c3cb82f-a848-44fe-8569-8c084df440f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '144025f3e2aa498f9b6eb4787e860159'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:26.848089 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1408/2814] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:57:26 2022] DELETE /identity/v3/users/144025f3e2aa498f9b6eb4787e860159 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:27.229574 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802dd8ae-c878-4fc1-a00d-3c207e15eaa7 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:27.231058 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-802dd8ae-c878-4fc1-a00d-3c207e15eaa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:27.245055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802dd8ae-c878-4fc1-a00d-3c207e15eaa7 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:27.271629 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:27.303603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b26283-6d28-482b-b598-80c895a4295d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:27.304455 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b26283-6d28-482b-b598-80c895a4295d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:27.304735 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b26283-6d28-482b-b598-80c895a4295d admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:27.305009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b26283-6d28-482b-b598-80c895a4295d admin admin] PATH_INFO: `/v3/projects/b5662afebfe64aeab2973b5b6ec7a7be` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:27.306156 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b26283-6d28-482b-b598-80c895a4295d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5662afebfe64aeab2973b5b6ec7a7be)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:27.308094 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b26283-6d28-482b-b598-80c895a4295d admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:27.410440 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b7b26283-6d28-482b-b598-80c895a4295d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5662afebfe64aeab2973b5b6ec7a7be', 'request_id': 'req-b7b26283-6d28-482b-b598-80c895a4295d'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:27.469683 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b7b26283-6d28-482b-b598-80c895a4295d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5662afebfe64aeab2973b5b6ec7a7be', 'request_id': 'req-b7b26283-6d28-482b-b598-80c895a4295d'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:27.476137 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b7b26283-6d28-482b-b598-80c895a4295d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5662afebfe64aeab2973b5b6ec7a7be', 'request_id': 'req-b7b26283-6d28-482b-b598-80c895a4295d'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:27.486409 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b7b26283-6d28-482b-b598-80c895a4295d admin admin] The token cache is being invalidate because project b5662afebfe64aeab2973b5b6ec7a7be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:57:27.486694 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b7b26283-6d28-482b-b598-80c895a4295d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:27.487341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-b7b26283-6d28-482b-b598-80c895a4295d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:27.488867 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1407/2815] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:57:27 2022] DELETE /identity/v3/projects/b5662afebfe64aeab2973b5b6ec7a7be => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:27.493939 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de0bf9-9e5a-4b14-af04-604305621f35 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:27.496631 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de0bf9-9e5a-4b14-af04-604305621f35 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:27.496631 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de0bf9-9e5a-4b14-af04-604305621f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:27.513193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-a8de0bf9-9e5a-4b14-af04-604305621f35 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:27.514470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-a8de0bf9-9e5a-4b14-af04-604305621f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:27.572687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1409/2816] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:27.580948 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e8f57-b4e2-4743-89b6-d583050200aa None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:27.581233 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e8f57-b4e2-4743-89b6-d583050200aa None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:27.581506 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e8f57-b4e2-4743-89b6-d583050200aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:27.599278 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-481e8f57-b4e2-4743-89b6-d583050200aa None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:27.600469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-481e8f57-b4e2-4743-89b6-d583050200aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:27.608270 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1408/2817] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:27.612393 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de0bf9-9e5a-4b14-af04-604305621f35 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:27.622025 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de0bf9-9e5a-4b14-af04-604305621f35 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:27.627343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b980a2d-d266-4b93-b04c-d9f0b407ea91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:27.628051 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b980a2d-d266-4b93-b04c-d9f0b407ea91 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:27.628332 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b980a2d-d266-4b93-b04c-d9f0b407ea91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:27.628604 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b980a2d-d266-4b93-b04c-d9f0b407ea91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:27.629323 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b980a2d-d266-4b93-b04c-d9f0b407ea91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:57:27.629632 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b980a2d-d266-4b93-b04c-d9f0b407ea91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:27.631242 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b980a2d-d266-4b93-b04c-d9f0b407ea91 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:27.643282 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1410/2818] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:57:27.647588 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e8f57-b4e2-4743-89b6-d583050200aa None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:27.656690 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e8f57-b4e2-4743-89b6-d583050200aa None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:27.661312 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdd712-1d0c-4731-938a-46a7322eec03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:27.661946 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdd712-1d0c-4731-938a-46a7322eec03 admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:27.662228 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdd712-1d0c-4731-938a-46a7322eec03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:27.662553 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdd712-1d0c-4731-938a-46a7322eec03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:27.663246 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdd712-1d0c-4731-938a-46a7322eec03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:27.665130 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdd712-1d0c-4731-938a-46a7322eec03 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:27.733555 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1409/2819] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:57:27 2022] POST /identity/v3/projects => generated 379 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:57:27.738192 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b980a2d-d266-4b93-b04c-d9f0b407ea91 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:27.748607 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b980a2d-d266-4b93-b04c-d9f0b407ea91 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:27.804528 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746bfa2c-243c-4298-9c8f-015da7b08eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:27.805183 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bfa2c-243c-4298-9c8f-015da7b08eea admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:27.805461 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bfa2c-243c-4298-9c8f-015da7b08eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:27.805734 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bfa2c-243c-4298-9c8f-015da7b08eea admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:27.806442 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746bfa2c-243c-4298-9c8f-015da7b08eea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:27.808344 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746bfa2c-243c-4298-9c8f-015da7b08eea admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:27.893812 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:27.898818 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1411/2820] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:57:27 2022] POST /identity/v3/users => generated 290 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:57:27.903120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdd712-1d0c-4731-938a-46a7322eec03 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:27.913618 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdd712-1d0c-4731-938a-46a7322eec03 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:27.918315 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9d0ec-25e1-42f3-9192-23f440efc763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:27.918992 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9d0ec-25e1-42f3-9192-23f440efc763 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:27.919277 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9d0ec-25e1-42f3-9192-23f440efc763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:27.919552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9d0ec-25e1-42f3-9192-23f440efc763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:27.920274 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9d0ec-25e1-42f3-9192-23f440efc763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:27.921863 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9d0ec-25e1-42f3-9192-23f440efc763 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:27.933912 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1410/2821] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:57:27.937019 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746bfa2c-243c-4298-9c8f-015da7b08eea admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:27.948193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746bfa2c-243c-4298-9c8f-015da7b08eea admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:27.952986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae54209-1c9f-4f69-b5a2-892ea2d77dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:27.953632 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae54209-1c9f-4f69-b5a2-892ea2d77dad admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:27.953915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae54209-1c9f-4f69-b5a2-892ea2d77dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:27.954194 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae54209-1c9f-4f69-b5a2-892ea2d77dad admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:27.954983 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae54209-1c9f-4f69-b5a2-892ea2d77dad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:27.956571 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae54209-1c9f-4f69-b5a2-892ea2d77dad admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:27.967550 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1412/2822] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:57:27.971973 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9d0ec-25e1-42f3-9192-23f440efc763 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:27.981894 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9d0ec-25e1-42f3-9192-23f440efc763 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:27.987273 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e882f9e-a25b-4062-b3bf-987d60590c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:27.988130 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e882f9e-a25b-4062-b3bf-987d60590c60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:27.988415 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e882f9e-a25b-4062-b3bf-987d60590c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:27.988695 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e882f9e-a25b-4062-b3bf-987d60590c60 admin admin] PATH_INFO: `/v3/projects/686a7c818e564703a909d11e274ac751/users/d0c28bc7f9374ac5a1b8cf1e96785881/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:28.003933 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e882f9e-a25b-4062-b3bf-987d60590c60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=686a7c818e564703a909d11e274ac751, user_id=d0c28bc7f9374ac5a1b8cf1e96785881, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:28.006655 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e882f9e-a25b-4062-b3bf-987d60590c60 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:28.087210 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1411/2823] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:57:27 2022] PUT /identity/v3/projects/686a7c818e564703a909d11e274ac751/users/d0c28bc7f9374ac5a1b8cf1e96785881/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:28.098108 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e60bb-f48e-4c04-a1a5-6109fba86612 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:28.098424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e60bb-f48e-4c04-a1a5-6109fba86612 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:28.098709 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e60bb-f48e-4c04-a1a5-6109fba86612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:28.124738 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-027e60bb-f48e-4c04-a1a5-6109fba86612 None None] MFA Rules not processed for user `d0c28bc7f9374ac5a1b8cf1e96785881`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:28.126191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-027e60bb-f48e-4c04-a1a5-6109fba86612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:28.189954 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1413/2824] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 18:57:28.194300 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e882f9e-a25b-4062-b3bf-987d60590c60 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:28.246847 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:28.257436 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e882f9e-a25b-4062-b3bf-987d60590c60 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:28.262081 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582d158-9781-4ce8-8ff1-f1fcdd6d9e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:28.262759 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582d158-9781-4ce8-8ff1-f1fcdd6d9e9d admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:28.263039 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582d158-9781-4ce8-8ff1-f1fcdd6d9e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:28.263311 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582d158-9781-4ce8-8ff1-f1fcdd6d9e9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:28.263999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582d158-9781-4ce8-8ff1-f1fcdd6d9e9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:28.265918 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582d158-9781-4ce8-8ff1-f1fcdd6d9e9d admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:28.339445 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1412/2825] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:57:28 2022] POST /identity/v3/projects => generated 381 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:57:28.343423 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e60bb-f48e-4c04-a1a5-6109fba86612 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:28.355062 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e60bb-f48e-4c04-a1a5-6109fba86612 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:28.411914 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622bce28-b662-4dd5-a915-e8f73750d4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:28.412667 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bce28-b662-4dd5-a915-e8f73750d4a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:28.412950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bce28-b662-4dd5-a915-e8f73750d4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:28.413226 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bce28-b662-4dd5-a915-e8f73750d4a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:28.413901 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622bce28-b662-4dd5-a915-e8f73750d4a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:28.415851 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622bce28-b662-4dd5-a915-e8f73750d4a8 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:28.503604 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1414/2826] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:57:28 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:57:28.507352 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582d158-9781-4ce8-8ff1-f1fcdd6d9e9d admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:28.518443 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582d158-9781-4ce8-8ff1-f1fcdd6d9e9d admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:28.523071 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f5921-6f1e-4b5a-ae43-a33db1128f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:28.524018 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f5921-6f1e-4b5a-ae43-a33db1128f52 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:28.524309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f5921-6f1e-4b5a-ae43-a33db1128f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:28.524584 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f5921-6f1e-4b5a-ae43-a33db1128f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:28.525308 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f5921-6f1e-4b5a-ae43-a33db1128f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:28.526939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f5921-6f1e-4b5a-ae43-a33db1128f52 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:28.538175 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1413/2827] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:57:28.542343 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622bce28-b662-4dd5-a915-e8f73750d4a8 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:28.552394 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622bce28-b662-4dd5-a915-e8f73750d4a8 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:28.556976 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e2c60-a97a-4efd-859b-9033d52cb12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:28.557851 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e2c60-a97a-4efd-859b-9033d52cb12f admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:28.558152 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e2c60-a97a-4efd-859b-9033d52cb12f admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:28.558449 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e2c60-a97a-4efd-859b-9033d52cb12f admin admin] PATH_INFO: `/v3/projects/38cbc8b9eb5947fb94bcfff7604f7a05/users/df945195bab642c1a9b7d772990d505f/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:28.573139 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e2c60-a97a-4efd-859b-9033d52cb12f admin admin] RBAC: Authorizing `identity:create_grant(project_id=38cbc8b9eb5947fb94bcfff7604f7a05, user_id=df945195bab642c1a9b7d772990d505f, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:28.575847 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e2c60-a97a-4efd-859b-9033d52cb12f admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:28.633655 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1415/2828] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:57:28 2022] PUT /identity/v3/projects/38cbc8b9eb5947fb94bcfff7604f7a05/users/df945195bab642c1a9b7d772990d505f/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:28.637617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f5921-6f1e-4b5a-ae43-a33db1128f52 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:28.648370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f5921-6f1e-4b5a-ae43-a33db1128f52 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:28.703405 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784fa88-03f9-4f0c-9892-602b232c4524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:28.704055 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784fa88-03f9-4f0c-9892-602b232c4524 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:28.704337 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784fa88-03f9-4f0c-9892-602b232c4524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:28.705162 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784fa88-03f9-4f0c-9892-602b232c4524 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:28.706080 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1784fa88-03f9-4f0c-9892-602b232c4524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:28.707700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1784fa88-03f9-4f0c-9892-602b232c4524 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:28.717185 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1414/2829] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:57:28.722084 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e2c60-a97a-4efd-859b-9033d52cb12f admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:28.732303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e2c60-a97a-4efd-859b-9033d52cb12f admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:28.737079 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e21fa-4c58-4779-82c1-1569f4b20feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:28.737951 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e21fa-4c58-4779-82c1-1569f4b20feb admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:28.738232 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e21fa-4c58-4779-82c1-1569f4b20feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:28.738539 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e21fa-4c58-4779-82c1-1569f4b20feb admin admin] PATH_INFO: `/v3/projects/38cbc8b9eb5947fb94bcfff7604f7a05/users/df945195bab642c1a9b7d772990d505f/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:28.740589 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e21fa-4c58-4779-82c1-1569f4b20feb admin admin] RBAC: Authorizing `identity:create_grant(project_id=38cbc8b9eb5947fb94bcfff7604f7a05, user_id=df945195bab642c1a9b7d772990d505f, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:28.743320 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e21fa-4c58-4779-82c1-1569f4b20feb admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:28.754500 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1416/2830] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:57:28 2022] PUT /identity/v3/projects/38cbc8b9eb5947fb94bcfff7604f7a05/users/df945195bab642c1a9b7d772990d505f/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:28.765403 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289af4a-d958-467b-a382-8371463950b6 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:28.765686 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289af4a-d958-467b-a382-8371463950b6 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:28.765986 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289af4a-d958-467b-a382-8371463950b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:28.794672 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-6289af4a-d958-467b-a382-8371463950b6 None None] MFA Rules not processed for user `df945195bab642c1a9b7d772990d505f`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:28.795865 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-6289af4a-d958-467b-a382-8371463950b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:28.862941 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1415/2831] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:28.873554 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e21fa-4c58-4779-82c1-1569f4b20feb admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:28.875268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-a71e21fa-4c58-4779-82c1-1569f4b20feb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:28.910483 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:28.934526 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e21fa-4c58-4779-82c1-1569f4b20feb admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:28.938437 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949aa68a-755b-4399-9e31-ae151c48c5a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:28.939962 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949aa68a-755b-4399-9e31-ae151c48c5a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:28.940093 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949aa68a-755b-4399-9e31-ae151c48c5a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:28.940301 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949aa68a-755b-4399-9e31-ae151c48c5a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:28.950107 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949aa68a-755b-4399-9e31-ae151c48c5a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:28.953360 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949aa68a-755b-4399-9e31-ae151c48c5a5 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:28.961055 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1417/2832] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:57:28 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:57:29.443871 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289af4a-d958-467b-a382-8371463950b6 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:29.491812 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:29.506014 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289af4a-d958-467b-a382-8371463950b6 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:29.510675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffc038-3a28-482c-80d7-3f2d64a211a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:29.511536 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffc038-3a28-482c-80d7-3f2d64a211a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:29.511819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffc038-3a28-482c-80d7-3f2d64a211a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:29.512096 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffc038-3a28-482c-80d7-3f2d64a211a9 admin admin] PATH_INFO: `/v3/users/d0c28bc7f9374ac5a1b8cf1e96785881` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:29.513281 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddffc038-3a28-482c-80d7-3f2d64a211a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0c28bc7f9374ac5a1b8cf1e96785881)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:29.515308 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddffc038-3a28-482c-80d7-3f2d64a211a9 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:29.678178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-ddffc038-3a28-482c-80d7-3f2d64a211a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0c28bc7f9374ac5a1b8cf1e96785881'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:29.684982 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-ddffc038-3a28-482c-80d7-3f2d64a211a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0c28bc7f9374ac5a1b8cf1e96785881'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:29.685645 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-ddffc038-3a28-482c-80d7-3f2d64a211a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0c28bc7f9374ac5a1b8cf1e96785881'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:29.686250 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-ddffc038-3a28-482c-80d7-3f2d64a211a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0c28bc7f9374ac5a1b8cf1e96785881'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:29.698146 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-ddffc038-3a28-482c-80d7-3f2d64a211a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0c28bc7f9374ac5a1b8cf1e96785881'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:29.754341 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1416/2833] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:57:29 2022] DELETE /identity/v3/users/d0c28bc7f9374ac5a1b8cf1e96785881 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:29.758878 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed3044-f02f-43d5-b8da-b2f2bc748e26 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:29.759337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed3044-f02f-43d5-b8da-b2f2bc748e26 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:29.759436 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed3044-f02f-43d5-b8da-b2f2bc748e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:29.776707 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-05ed3044-f02f-43d5-b8da-b2f2bc748e26 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:29.777941 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-05ed3044-f02f-43d5-b8da-b2f2bc748e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:29.836608 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1418/2834] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:29.846811 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffc038-3a28-482c-80d7-3f2d64a211a9 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:29.848243 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ddffc038-3a28-482c-80d7-3f2d64a211a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:29.904933 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffc038-3a28-482c-80d7-3f2d64a211a9 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:29.909133 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e9a8e-f970-4610-96f9-547f32007bf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:29.909765 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e9a8e-f970-4610-96f9-547f32007bf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:29.910048 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e9a8e-f970-4610-96f9-547f32007bf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:29.910321 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e9a8e-f970-4610-96f9-547f32007bf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:29.919313 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e9a8e-f970-4610-96f9-547f32007bf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:29.920732 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e9a8e-f970-4610-96f9-547f32007bf7 service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:29.931048 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1417/2835] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:57:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:57:30.323223 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed3044-f02f-43d5-b8da-b2f2bc748e26 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:30.325930 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-05ed3044-f02f-43d5-b8da-b2f2bc748e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:30.337619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed3044-f02f-43d5-b8da-b2f2bc748e26 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:30.342204 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5bd25-4bc1-4e8f-9a3c-62d88730e7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:30.343128 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5bd25-4bc1-4e8f-9a3c-62d88730e7af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:30.343405 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5bd25-4bc1-4e8f-9a3c-62d88730e7af admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:30.343676 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5bd25-4bc1-4e8f-9a3c-62d88730e7af admin admin] PATH_INFO: `/v3/projects/686a7c818e564703a909d11e274ac751` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:30.344852 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa5bd25-4bc1-4e8f-9a3c-62d88730e7af admin admin] RBAC: Authorizing `identity:delete_project(project_id=686a7c818e564703a909d11e274ac751)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:30.347860 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa5bd25-4bc1-4e8f-9a3c-62d88730e7af admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:30.446769 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2fa5bd25-4bc1-4e8f-9a3c-62d88730e7af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '686a7c818e564703a909d11e274ac751', 'request_id': 'req-2fa5bd25-4bc1-4e8f-9a3c-62d88730e7af'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:30.515418 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2fa5bd25-4bc1-4e8f-9a3c-62d88730e7af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '686a7c818e564703a909d11e274ac751', 'request_id': 'req-2fa5bd25-4bc1-4e8f-9a3c-62d88730e7af'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:30.522217 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2fa5bd25-4bc1-4e8f-9a3c-62d88730e7af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '686a7c818e564703a909d11e274ac751', 'request_id': 'req-2fa5bd25-4bc1-4e8f-9a3c-62d88730e7af'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:30.532557 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2fa5bd25-4bc1-4e8f-9a3c-62d88730e7af admin admin] The token cache is being invalidate because project 686a7c818e564703a909d11e274ac751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:57:30.533337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2fa5bd25-4bc1-4e8f-9a3c-62d88730e7af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:30.533956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-2fa5bd25-4bc1-4e8f-9a3c-62d88730e7af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:30.535525 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1419/2836] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:57:30 2022] DELETE /identity/v3/projects/686a7c818e564703a909d11e274ac751 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:30.539441 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e9a8e-f970-4610-96f9-547f32007bf7 service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:30.550166 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e9a8e-f970-4610-96f9-547f32007bf7 service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:30.606016 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda45a5e-bbb6-4d80-b41a-704ca923d3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:30.606918 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda45a5e-bbb6-4d80-b41a-704ca923d3de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:30.607202 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda45a5e-bbb6-4d80-b41a-704ca923d3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:30.607482 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda45a5e-bbb6-4d80-b41a-704ca923d3de admin admin] PATH_INFO: `/v3/users/df945195bab642c1a9b7d772990d505f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:30.608668 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda45a5e-bbb6-4d80-b41a-704ca923d3de admin admin] RBAC: Authorizing `identity:delete_user(user_id=df945195bab642c1a9b7d772990d505f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:30.610608 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda45a5e-bbb6-4d80-b41a-704ca923d3de admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:30.623450 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:30.790963 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-cda45a5e-bbb6-4d80-b41a-704ca923d3de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df945195bab642c1a9b7d772990d505f'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:30.797774 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-cda45a5e-bbb6-4d80-b41a-704ca923d3de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df945195bab642c1a9b7d772990d505f'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:30.798794 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-cda45a5e-bbb6-4d80-b41a-704ca923d3de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df945195bab642c1a9b7d772990d505f'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:30.799381 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-cda45a5e-bbb6-4d80-b41a-704ca923d3de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df945195bab642c1a9b7d772990d505f'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:30.811004 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-cda45a5e-bbb6-4d80-b41a-704ca923d3de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df945195bab642c1a9b7d772990d505f'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:30.898046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1418/2837] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:57:30 2022] DELETE /identity/v3/users/df945195bab642c1a9b7d772990d505f => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 18:57:31.282907 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5bd25-4bc1-4e8f-9a3c-62d88730e7af admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:31.284376 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-2fa5bd25-4bc1-4e8f-9a3c-62d88730e7af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:31.300012 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5bd25-4bc1-4e8f-9a3c-62d88730e7af admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:31.354249 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:31.360621 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dbb8a1-7caf-405a-a02a-7fbcd13afd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:31.361654 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbb8a1-7caf-405a-a02a-7fbcd13afd80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:31.361956 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbb8a1-7caf-405a-a02a-7fbcd13afd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:31.362232 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dbb8a1-7caf-405a-a02a-7fbcd13afd80 admin admin] PATH_INFO: `/v3/projects/38cbc8b9eb5947fb94bcfff7604f7a05` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:31.363488 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dbb8a1-7caf-405a-a02a-7fbcd13afd80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38cbc8b9eb5947fb94bcfff7604f7a05)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:31.365400 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dbb8a1-7caf-405a-a02a-7fbcd13afd80 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:31.453637 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-53dbb8a1-7caf-405a-a02a-7fbcd13afd80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38cbc8b9eb5947fb94bcfff7604f7a05', 'request_id': 'req-53dbb8a1-7caf-405a-a02a-7fbcd13afd80'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:31.509684 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-53dbb8a1-7caf-405a-a02a-7fbcd13afd80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38cbc8b9eb5947fb94bcfff7604f7a05', 'request_id': 'req-53dbb8a1-7caf-405a-a02a-7fbcd13afd80'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:31.516257 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-53dbb8a1-7caf-405a-a02a-7fbcd13afd80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38cbc8b9eb5947fb94bcfff7604f7a05', 'request_id': 'req-53dbb8a1-7caf-405a-a02a-7fbcd13afd80'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:31.529961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-53dbb8a1-7caf-405a-a02a-7fbcd13afd80 admin admin] The token cache is being invalidate because project 38cbc8b9eb5947fb94bcfff7604f7a05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80569) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:57:31.529961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-53dbb8a1-7caf-405a-a02a-7fbcd13afd80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:31.529961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-53dbb8a1-7caf-405a-a02a-7fbcd13afd80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:31.529961 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1420/2838] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:57:31 2022] DELETE /identity/v3/projects/38cbc8b9eb5947fb94bcfff7604f7a05 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:31.534364 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5a50b-87b4-434a-8bea-5223cd598015 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:31.534716 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5a50b-87b4-434a-8bea-5223cd598015 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:31.534999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5a50b-87b4-434a-8bea-5223cd598015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:31.553481 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-40c5a50b-87b4-434a-8bea-5223cd598015 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:31.554715 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-40c5a50b-87b4-434a-8bea-5223cd598015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:31.613633 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1419/2839] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:31.621603 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39673117-7277-4068-aed8-58d8e5c89782 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:31.621887 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39673117-7277-4068-aed8-58d8e5c89782 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:31.622162 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39673117-7277-4068-aed8-58d8e5c89782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:31.640356 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-39673117-7277-4068-aed8-58d8e5c89782 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:31.641549 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-39673117-7277-4068-aed8-58d8e5c89782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:31.649909 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1421/2840] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:31.654009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5a50b-87b4-434a-8bea-5223cd598015 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:31.665042 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5a50b-87b4-434a-8bea-5223cd598015 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:31.669522 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6aed6f-c6c7-4436-822c-3e1b4d0f241a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:31.670166 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6aed6f-c6c7-4436-822c-3e1b4d0f241a admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:31.670551 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6aed6f-c6c7-4436-822c-3e1b4d0f241a admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:31.670829 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6aed6f-c6c7-4436-822c-3e1b4d0f241a admin admin] PATH_INFO: `/v3/domains` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:31.671552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6aed6f-c6c7-4436-822c-3e1b4d0f241a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:57:31.671857 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6aed6f-c6c7-4436-822c-3e1b4d0f241a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:31.673468 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6aed6f-c6c7-4436-822c-3e1b4d0f241a admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:31.684711 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1420/2841] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:57:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:57:31.688906 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39673117-7277-4068-aed8-58d8e5c89782 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:31.697910 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39673117-7277-4068-aed8-58d8e5c89782 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:31.702536 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2091b-da2b-455b-856b-c213952457ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:31.703166 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2091b-da2b-455b-856b-c213952457ab admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:31.703470 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2091b-da2b-455b-856b-c213952457ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:31.703740 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2091b-da2b-455b-856b-c213952457ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:31.704411 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2091b-da2b-455b-856b-c213952457ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:31.707180 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2091b-da2b-455b-856b-c213952457ab admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:31.791727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1422/2842] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:57:31 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:57:31.796129 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6aed6f-c6c7-4436-822c-3e1b4d0f241a admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:31.807263 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6aed6f-c6c7-4436-822c-3e1b4d0f241a admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:31.866160 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8541ebb-715f-4e18-96b2-5ef648bf2b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:31.866920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8541ebb-715f-4e18-96b2-5ef648bf2b7d admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:31.867203 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8541ebb-715f-4e18-96b2-5ef648bf2b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:31.867480 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8541ebb-715f-4e18-96b2-5ef648bf2b7d admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:31.868160 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8541ebb-715f-4e18-96b2-5ef648bf2b7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:31.870134 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8541ebb-715f-4e18-96b2-5ef648bf2b7d admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:31.944622 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1421/2843] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:57:31 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:57:31.948695 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2091b-da2b-455b-856b-c213952457ab admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:31.958985 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2091b-da2b-455b-856b-c213952457ab admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:31.963536 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0785ba-322b-4893-9501-ebb923955408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:31.964179 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0785ba-322b-4893-9501-ebb923955408 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:31.964460 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0785ba-322b-4893-9501-ebb923955408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:31.964735 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0785ba-322b-4893-9501-ebb923955408 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:31.965451 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0785ba-322b-4893-9501-ebb923955408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:31.967086 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0785ba-322b-4893-9501-ebb923955408 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:31.978031 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1423/2844] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:57:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:57:31.991099 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8541ebb-715f-4e18-96b2-5ef648bf2b7d admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:32.001553 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8541ebb-715f-4e18-96b2-5ef648bf2b7d admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:32.006274 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52504739-7f5b-4bdb-80d3-0b48e2f58f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:32.006947 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52504739-7f5b-4bdb-80d3-0b48e2f58f99 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:32.007228 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52504739-7f5b-4bdb-80d3-0b48e2f58f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:32.007500 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52504739-7f5b-4bdb-80d3-0b48e2f58f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:32.008216 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52504739-7f5b-4bdb-80d3-0b48e2f58f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:32.009786 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52504739-7f5b-4bdb-80d3-0b48e2f58f99 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:32.011024 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:32.020344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1422/2845] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:57:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:57:32.025577 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0785ba-322b-4893-9501-ebb923955408 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:32.036586 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0785ba-322b-4893-9501-ebb923955408 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:32.041202 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d29701-4d32-463c-b90d-d6e793844533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:32.042115 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d29701-4d32-463c-b90d-d6e793844533 admin admin] REQUEST_METHOD: `PUT` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:32.042337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d29701-4d32-463c-b90d-d6e793844533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:32.042635 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d29701-4d32-463c-b90d-d6e793844533 admin admin] PATH_INFO: `/v3/projects/9f723be4fa7e4d02b7ba5627a34329dd/users/ade274a1060f4f269f619e0d80da142c/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:32.057934 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d29701-4d32-463c-b90d-d6e793844533 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f723be4fa7e4d02b7ba5627a34329dd, user_id=ade274a1060f4f269f619e0d80da142c, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:32.060582 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d29701-4d32-463c-b90d-d6e793844533 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:32.125886 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1424/2846] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:57:32 2022] PUT /identity/v3/projects/9f723be4fa7e4d02b7ba5627a34329dd/users/ade274a1060f4f269f619e0d80da142c/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:32.131020 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa21877-4fa2-4bf5-b47d-6336218cf7fe None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:32.131326 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa21877-4fa2-4bf5-b47d-6336218cf7fe None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:32.131604 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa21877-4fa2-4bf5-b47d-6336218cf7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:32.149433 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-6aa21877-4fa2-4bf5-b47d-6336218cf7fe None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:32.150676 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-6aa21877-4fa2-4bf5-b47d-6336218cf7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:32.209798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1423/2847] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:32.219353 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d29701-4d32-463c-b90d-d6e793844533 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:32.221643 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-42d29701-4d32-463c-b90d-d6e793844533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:32.278107 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d29701-4d32-463c-b90d-d6e793844533 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:32.281737 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263e5fe-4fca-44b6-9f65-c16284c158dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:32.282365 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263e5fe-4fca-44b6-9f65-c16284c158dc service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:32.283233 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263e5fe-4fca-44b6-9f65-c16284c158dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:32.283815 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263e5fe-4fca-44b6-9f65-c16284c158dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:32.292676 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263e5fe-4fca-44b6-9f65-c16284c158dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:32.294044 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263e5fe-4fca-44b6-9f65-c16284c158dc service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:32.303713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1425/2848] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:57:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:57:33.419225 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ae404-7098-4579-91c3-d2082d329fac None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:33.419798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ae404-7098-4579-91c3-d2082d329fac None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:33.419798 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ae404-7098-4579-91c3-d2082d329fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:33.447868 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-b11ae404-7098-4579-91c3-d2082d329fac None None] MFA Rules not processed for user `ade274a1060f4f269f619e0d80da142c`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:33.449378 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-b11ae404-7098-4579-91c3-d2082d329fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:33.519764 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1424/2849] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:33.523953 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263e5fe-4fca-44b6-9f65-c16284c158dc service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:33.534341 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263e5fe-4fca-44b6-9f65-c16284c158dc service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:33.539417 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ab4a5-312a-42e3-9f43-e3d4df3c831c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:33.543919 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ab4a5-312a-42e3-9f43-e3d4df3c831c admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:33.543919 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ab4a5-312a-42e3-9f43-e3d4df3c831c admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:33.543919 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ab4a5-312a-42e3-9f43-e3d4df3c831c admin admin] PATH_INFO: `/v3/projects` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:33.543919 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ab4a5-312a-42e3-9f43-e3d4df3c831c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:33.543919 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ab4a5-312a-42e3-9f43-e3d4df3c831c admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:33.643876 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1426/2850] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:57:33 2022] POST /identity/v3/projects => generated 371 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:57:33.647793 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ae404-7098-4579-91c3-d2082d329fac None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:33.659160 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ae404-7098-4579-91c3-d2082d329fac None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:33.714122 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82133643-9fa5-4a81-905c-67155794dd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:33.714821 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82133643-9fa5-4a81-905c-67155794dd1d admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:33.715103 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82133643-9fa5-4a81-905c-67155794dd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:33.715399 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82133643-9fa5-4a81-905c-67155794dd1d admin admin] PATH_INFO: `/v3/users` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:33.716054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82133643-9fa5-4a81-905c-67155794dd1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:33.717969 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82133643-9fa5-4a81-905c-67155794dd1d admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:33.795919 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1425/2851] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:57:33 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:57:33.799987 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ab4a5-312a-42e3-9f43-e3d4df3c831c admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:33.806613 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:33.810274 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ab4a5-312a-42e3-9f43-e3d4df3c831c admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:33.814986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58903511-9075-4133-b1b3-6bf78b8def70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:33.815632 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58903511-9075-4133-b1b3-6bf78b8def70 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:33.819248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58903511-9075-4133-b1b3-6bf78b8def70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:33.819248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58903511-9075-4133-b1b3-6bf78b8def70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:33.819248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58903511-9075-4133-b1b3-6bf78b8def70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:33.819248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58903511-9075-4133-b1b3-6bf78b8def70 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:33.829275 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1427/2852] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:57:33.834552 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82133643-9fa5-4a81-905c-67155794dd1d admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:33.844553 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82133643-9fa5-4a81-905c-67155794dd1d admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:33.850453 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f27da-40a9-46c2-8996-000607af5e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:33.851311 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f27da-40a9-46c2-8996-000607af5e45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:33.851622 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f27da-40a9-46c2-8996-000607af5e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:33.851931 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f27da-40a9-46c2-8996-000607af5e45 admin admin] PATH_INFO: `/v3/projects/4806024a5ee8470b9567e283c658dd61/users/ffdcf38999d542608c5e654a6e53afaa/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:33.864493 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:33.867540 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f27da-40a9-46c2-8996-000607af5e45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4806024a5ee8470b9567e283c658dd61, user_id=ffdcf38999d542608c5e654a6e53afaa, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:33.870323 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f27da-40a9-46c2-8996-000607af5e45 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:33.944541 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1426/2853] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:57:33 2022] PUT /identity/v3/projects/4806024a5ee8470b9567e283c658dd61/users/ffdcf38999d542608c5e654a6e53afaa/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:33.948596 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58903511-9075-4133-b1b3-6bf78b8def70 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:33.959668 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58903511-9075-4133-b1b3-6bf78b8def70 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:34.016686 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b61d2-e499-4f57-9c5d-b958abed45bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:34.018007 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b61d2-e499-4f57-9c5d-b958abed45bd admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:34.018292 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b61d2-e499-4f57-9c5d-b958abed45bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:34.018597 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b61d2-e499-4f57-9c5d-b958abed45bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:34.019325 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b61d2-e499-4f57-9c5d-b958abed45bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:34.020900 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b61d2-e499-4f57-9c5d-b958abed45bd admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:34.030485 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1428/2854] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:57:34.035772 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f27da-40a9-46c2-8996-000607af5e45 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:34.045925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f27da-40a9-46c2-8996-000607af5e45 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:34.051112 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebb039-a9c8-4f13-a117-d7399f0424e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:34.051973 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebb039-a9c8-4f13-a117-d7399f0424e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:34.052248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebb039-a9c8-4f13-a117-d7399f0424e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:34.052516 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebb039-a9c8-4f13-a117-d7399f0424e5 admin admin] PATH_INFO: `/v3/projects/4806024a5ee8470b9567e283c658dd61/users/ffdcf38999d542608c5e654a6e53afaa/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:34.054508 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ebb039-a9c8-4f13-a117-d7399f0424e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4806024a5ee8470b9567e283c658dd61, user_id=ffdcf38999d542608c5e654a6e53afaa, role_id=ff7e725e9b7a4f2fbbb8ca8c518b208f)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:34.057139 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ebb039-a9c8-4f13-a117-d7399f0424e5 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:34.068280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1427/2855] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:57:34 2022] PUT /identity/v3/projects/4806024a5ee8470b9567e283c658dd61/users/ffdcf38999d542608c5e654a6e53afaa/roles/ff7e725e9b7a4f2fbbb8ca8c518b208f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:35.207897 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5bb674-8a49-46ae-bdf7-e3c47b0749b6 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:35.208492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5bb674-8a49-46ae-bdf7-e3c47b0749b6 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:35.208673 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5bb674-8a49-46ae-bdf7-e3c47b0749b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:35.237144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-5a5bb674-8a49-46ae-bdf7-e3c47b0749b6 None None] MFA Rules not processed for user `ffdcf38999d542608c5e654a6e53afaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:35.238490 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-5a5bb674-8a49-46ae-bdf7-e3c47b0749b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:35.311559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1429/2856] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:35 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:35.322919 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebb039-a9c8-4f13-a117-d7399f0424e5 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:35.324608 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-43ebb039-a9c8-4f13-a117-d7399f0424e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:35.381083 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebb039-a9c8-4f13-a117-d7399f0424e5 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:35.384653 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ade41-224e-48cd-9cf8-8682f8ba232d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:35.385282 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ade41-224e-48cd-9cf8-8682f8ba232d service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:35.385560 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ade41-224e-48cd-9cf8-8682f8ba232d service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:35.385828 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ade41-224e-48cd-9cf8-8682f8ba232d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:35.440671 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ade41-224e-48cd-9cf8-8682f8ba232d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:35.442046 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ade41-224e-48cd-9cf8-8682f8ba232d service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:35.451021 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1428/2857] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:57:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:57:37.313413 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5bb674-8a49-46ae-bdf7-e3c47b0749b6 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:37.315448 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-5a5bb674-8a49-46ae-bdf7-e3c47b0749b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:37.327145 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5bb674-8a49-46ae-bdf7-e3c47b0749b6 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:37.331329 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701beae-b13f-48fb-8690-69f8fc4283e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:37.331957 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701beae-b13f-48fb-8690-69f8fc4283e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:37.332234 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701beae-b13f-48fb-8690-69f8fc4283e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:37.332504 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701beae-b13f-48fb-8690-69f8fc4283e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:37.341547 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a701beae-b13f-48fb-8690-69f8fc4283e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:37.342997 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a701beae-b13f-48fb-8690-69f8fc4283e5 service cinder] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:37.352269 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1430/2858] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:57:37 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:57:45.446721 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=56,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:47.348061 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=72 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:52.744232 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ade41-224e-48cd-9cf8-8682f8ba232d service cinder] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:52.745819 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-a91ade41-224e-48cd-9cf8-8682f8ba232d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:52.828608 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ade41-224e-48cd-9cf8-8682f8ba232d service cinder] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:52.834698 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22f733-0b49-496f-9b1f-01130fac1741 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cd6f0a1286243e4add692db01028205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:52.835339 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22f733-0b49-496f-9b1f-01130fac1741 service placement] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:52.835623 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22f733-0b49-496f-9b1f-01130fac1741 service placement] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:52.835896 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22f733-0b49-496f-9b1f-01130fac1741 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:52.837643 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ce22f733-0b49-496f-9b1f-01130fac1741 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:52.849225 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce22f733-0b49-496f-9b1f-01130fac1741 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:52.850934 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce22f733-0b49-496f-9b1f-01130fac1741 service placement] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:52.861075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1429/2859] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:57:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:57:52.880280 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701beae-b13f-48fb-8690-69f8fc4283e5 service cinder] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:52.946015 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701beae-b13f-48fb-8690-69f8fc4283e5 service cinder] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:52.950543 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e870c9-1672-4faa-a82e-3b1a29908ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:52.951402 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e870c9-1672-4faa-a82e-3b1a29908ab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:52.951684 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e870c9-1672-4faa-a82e-3b1a29908ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:52.951968 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e870c9-1672-4faa-a82e-3b1a29908ab4 admin admin] PATH_INFO: `/v3/users/ade274a1060f4f269f619e0d80da142c` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:52.953136 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e870c9-1672-4faa-a82e-3b1a29908ab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ade274a1060f4f269f619e0d80da142c)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:52.955117 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e870c9-1672-4faa-a82e-3b1a29908ab4 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:53.140970 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-31e870c9-1672-4faa-a82e-3b1a29908ab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ade274a1060f4f269f619e0d80da142c'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:53.148614 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-31e870c9-1672-4faa-a82e-3b1a29908ab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ade274a1060f4f269f619e0d80da142c'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:53.149290 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-31e870c9-1672-4faa-a82e-3b1a29908ab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ade274a1060f4f269f619e0d80da142c'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:53.161130 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-31e870c9-1672-4faa-a82e-3b1a29908ab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ade274a1060f4f269f619e0d80da142c'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:53.229173 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-31e870c9-1672-4faa-a82e-3b1a29908ab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ade274a1060f4f269f619e0d80da142c'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:53.235304 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1431/2860] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:57:52 2022] DELETE /identity/v3/users/ade274a1060f4f269f619e0d80da142c => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:53.465458 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22f733-0b49-496f-9b1f-01130fac1741 service placement] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:53.476931 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22f733-0b49-496f-9b1f-01130fac1741 service placement] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:53.537117 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593a5fd-f742-4575-9b80-9f1c85c3e853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:53.537975 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593a5fd-f742-4575-9b80-9f1c85c3e853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:53.538263 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593a5fd-f742-4575-9b80-9f1c85c3e853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:53.538587 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593a5fd-f742-4575-9b80-9f1c85c3e853 admin admin] PATH_INFO: `/v3/projects/9f723be4fa7e4d02b7ba5627a34329dd` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:53.539766 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593a5fd-f742-4575-9b80-9f1c85c3e853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f723be4fa7e4d02b7ba5627a34329dd)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:53.541675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593a5fd-f742-4575-9b80-9f1c85c3e853 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:53.628309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f593a5fd-f742-4575-9b80-9f1c85c3e853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f723be4fa7e4d02b7ba5627a34329dd', 'request_id': 'req-f593a5fd-f742-4575-9b80-9f1c85c3e853'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:53.691051 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f593a5fd-f742-4575-9b80-9f1c85c3e853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f723be4fa7e4d02b7ba5627a34329dd', 'request_id': 'req-f593a5fd-f742-4575-9b80-9f1c85c3e853'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:53.697517 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f593a5fd-f742-4575-9b80-9f1c85c3e853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f723be4fa7e4d02b7ba5627a34329dd', 'request_id': 'req-f593a5fd-f742-4575-9b80-9f1c85c3e853'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:53.708337 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f593a5fd-f742-4575-9b80-9f1c85c3e853 admin admin] The token cache is being invalidate because project 9f723be4fa7e4d02b7ba5627a34329dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:57:53.708618 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f593a5fd-f742-4575-9b80-9f1c85c3e853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:53.709238 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-f593a5fd-f742-4575-9b80-9f1c85c3e853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:53.710823 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1430/2861] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:57:53 2022] DELETE /identity/v3/projects/9f723be4fa7e4d02b7ba5627a34329dd => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:53.714840 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e870c9-1672-4faa-a82e-3b1a29908ab4 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:53.716240 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-31e870c9-1672-4faa-a82e-3b1a29908ab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:53.730029 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e870c9-1672-4faa-a82e-3b1a29908ab4 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:53.789687 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd4c4d-4434-45f9-aa56-847b2bea793c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:53.790586 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd4c4d-4434-45f9-aa56-847b2bea793c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:53.790866 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd4c4d-4434-45f9-aa56-847b2bea793c admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:53.791134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd4c4d-4434-45f9-aa56-847b2bea793c admin admin] PATH_INFO: `/v3/users/ffdcf38999d542608c5e654a6e53afaa` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:53.792297 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bd4c4d-4434-45f9-aa56-847b2bea793c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffdcf38999d542608c5e654a6e53afaa)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:53.794199 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bd4c4d-4434-45f9-aa56-847b2bea793c admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:53.815972 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:54.031580 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-82bd4c4d-4434-45f9-aa56-847b2bea793c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffdcf38999d542608c5e654a6e53afaa'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:54.039517 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-82bd4c4d-4434-45f9-aa56-847b2bea793c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffdcf38999d542608c5e654a6e53afaa'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:54.040193 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-82bd4c4d-4434-45f9-aa56-847b2bea793c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffdcf38999d542608c5e654a6e53afaa'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:54.052261 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-82bd4c4d-4434-45f9-aa56-847b2bea793c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffdcf38999d542608c5e654a6e53afaa'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:54.132147 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-82bd4c4d-4434-45f9-aa56-847b2bea793c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffdcf38999d542608c5e654a6e53afaa'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:54.138656 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1432/2862] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:57:53 2022] DELETE /identity/v3/users/ffdcf38999d542608c5e654a6e53afaa => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:54.357088 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593a5fd-f742-4575-9b80-9f1c85c3e853 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:54.359626 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-f593a5fd-f742-4575-9b80-9f1c85c3e853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:54.374723 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593a5fd-f742-4575-9b80-9f1c85c3e853 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:54.398009 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:54.438192 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0968f-35d7-4e51-b810-54a09565f60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:54.439075 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0968f-35d7-4e51-b810-54a09565f60e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:54.439355 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0968f-35d7-4e51-b810-54a09565f60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:54.439662 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0968f-35d7-4e51-b810-54a09565f60e admin admin] PATH_INFO: `/v3/projects/4806024a5ee8470b9567e283c658dd61` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:54.441955 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0968f-35d7-4e51-b810-54a09565f60e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4806024a5ee8470b9567e283c658dd61)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:54.443882 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0968f-35d7-4e51-b810-54a09565f60e admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:54.544230 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-32b0968f-35d7-4e51-b810-54a09565f60e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4806024a5ee8470b9567e283c658dd61', 'request_id': 'req-32b0968f-35d7-4e51-b810-54a09565f60e'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:54.600617 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-32b0968f-35d7-4e51-b810-54a09565f60e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4806024a5ee8470b9567e283c658dd61', 'request_id': 'req-32b0968f-35d7-4e51-b810-54a09565f60e'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:54.607073 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-32b0968f-35d7-4e51-b810-54a09565f60e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4806024a5ee8470b9567e283c658dd61', 'request_id': 'req-32b0968f-35d7-4e51-b810-54a09565f60e'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:54.618518 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-32b0968f-35d7-4e51-b810-54a09565f60e admin admin] The token cache is being invalidate because project 4806024a5ee8470b9567e283c658dd61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:57:54.618806 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-32b0968f-35d7-4e51-b810-54a09565f60e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:54.619441 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-32b0968f-35d7-4e51-b810-54a09565f60e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:54.620978 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1431/2863] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:57:54 2022] DELETE /identity/v3/projects/4806024a5ee8470b9567e283c658dd61 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:54.626267 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd8c4f-ec74-4f17-8c14-fc4bab9aa901 None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:54.626590 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd8c4f-ec74-4f17-8c14-fc4bab9aa901 None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:54.626869 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd8c4f-ec74-4f17-8c14-fc4bab9aa901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:54.656571 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-c5bd8c4f-ec74-4f17-8c14-fc4bab9aa901 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:54.657771 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-c5bd8c4f-ec74-4f17-8c14-fc4bab9aa901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:54.715006 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1433/2864] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:54.722933 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def73688-610c-476a-8e49-b755c8b85da5 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:54.723226 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def73688-610c-476a-8e49-b755c8b85da5 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:54.723509 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def73688-610c-476a-8e49-b755c8b85da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:54.741920 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-def73688-610c-476a-8e49-b755c8b85da5 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:54.743200 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-def73688-610c-476a-8e49-b755c8b85da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:54.751440 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1432/2865] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:54.756768 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd8c4f-ec74-4f17-8c14-fc4bab9aa901 None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:54.766558 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd8c4f-ec74-4f17-8c14-fc4bab9aa901 None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:54.771268 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18af867-d5c8-4ffd-98f5-1f9037741ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:54.771910 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18af867-d5c8-4ffd-98f5-1f9037741ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:54.772191 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18af867-d5c8-4ffd-98f5-1f9037741ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:54.772465 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18af867-d5c8-4ffd-98f5-1f9037741ef1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:54.773184 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18af867-d5c8-4ffd-98f5-1f9037741ef1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:57:54.773487 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18af867-d5c8-4ffd-98f5-1f9037741ef1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:54.775111 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18af867-d5c8-4ffd-98f5-1f9037741ef1 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:54.786607 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1434/2866] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:57:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:57:54.790694 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def73688-610c-476a-8e49-b755c8b85da5 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:54.799717 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def73688-610c-476a-8e49-b755c8b85da5 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:54.805232 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edccdcbf-f678-439e-b182-60c7a298c14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:54.805865 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccdcbf-f678-439e-b182-60c7a298c14e admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:54.806233 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccdcbf-f678-439e-b182-60c7a298c14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:54.806673 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccdcbf-f678-439e-b182-60c7a298c14e admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:54.807249 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edccdcbf-f678-439e-b182-60c7a298c14e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:54.809122 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edccdcbf-f678-439e-b182-60c7a298c14e admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:54.934525 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1433/2867] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:57:54 2022] POST /identity/v3/projects => generated 361 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:57:54.938291 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18af867-d5c8-4ffd-98f5-1f9037741ef1 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:54.949322 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18af867-d5c8-4ffd-98f5-1f9037741ef1 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:55.006022 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11bed55-1cbe-48bf-b3dc-3d395111c8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:55.006709 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bed55-1cbe-48bf-b3dc-3d395111c8c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:55.006993 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bed55-1cbe-48bf-b3dc-3d395111c8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:55.007262 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11bed55-1cbe-48bf-b3dc-3d395111c8c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:55.007940 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11bed55-1cbe-48bf-b3dc-3d395111c8c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:55.009846 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11bed55-1cbe-48bf-b3dc-3d395111c8c3 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:55.116363 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1435/2868] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:57:54 2022] POST /identity/v3/users => generated 281 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 18:57:55.120208 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edccdcbf-f678-439e-b182-60c7a298c14e admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:55.131100 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edccdcbf-f678-439e-b182-60c7a298c14e admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:55.135787 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803a7c9-dda3-4282-b577-e039fdecf020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:55.136432 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803a7c9-dda3-4282-b577-e039fdecf020 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:55.136720 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803a7c9-dda3-4282-b577-e039fdecf020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:55.137054 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803a7c9-dda3-4282-b577-e039fdecf020 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:55.137810 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6803a7c9-dda3-4282-b577-e039fdecf020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:55.139440 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6803a7c9-dda3-4282-b577-e039fdecf020 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:55.149838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1434/2869] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:57:55.153606 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11bed55-1cbe-48bf-b3dc-3d395111c8c3 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:55.164283 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11bed55-1cbe-48bf-b3dc-3d395111c8c3 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:55.169165 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869ae28-d97e-43d8-9887-6e5e1b11a8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:55.169827 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869ae28-d97e-43d8-9887-6e5e1b11a8a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:55.170115 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869ae28-d97e-43d8-9887-6e5e1b11a8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:55.170467 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869ae28-d97e-43d8-9887-6e5e1b11a8a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:55.171215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869ae28-d97e-43d8-9887-6e5e1b11a8a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:55.172838 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869ae28-d97e-43d8-9887-6e5e1b11a8a5 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:55.174119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:55.184727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1436/2870] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:57:55.188700 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803a7c9-dda3-4282-b577-e039fdecf020 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:55.198652 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803a7c9-dda3-4282-b577-e039fdecf020 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:55.203311 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0370a9-0d7f-4f93-a58b-1084d93f078a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:55.204178 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0370a9-0d7f-4f93-a58b-1084d93f078a admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:55.204470 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0370a9-0d7f-4f93-a58b-1084d93f078a admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:55.204748 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0370a9-0d7f-4f93-a58b-1084d93f078a admin admin] PATH_INFO: `/v3/projects/0c289a6f8ab1459ead2fb60cd28d103c/users/6738a8328dfb4345b61a4356cb09b8da/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:55.219097 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0370a9-0d7f-4f93-a58b-1084d93f078a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c289a6f8ab1459ead2fb60cd28d103c, user_id=6738a8328dfb4345b61a4356cb09b8da, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:55.222280 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0370a9-0d7f-4f93-a58b-1084d93f078a admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:55.292070 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1435/2871] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:57:55 2022] PUT /identity/v3/projects/0c289a6f8ab1459ead2fb60cd28d103c/users/6738a8328dfb4345b61a4356cb09b8da/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:55.304752 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38cf6c-6fa0-4655-ba28-ce2ff07d3c3d None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:55.305046 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38cf6c-6fa0-4655-ba28-ce2ff07d3c3d None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:55.305329 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38cf6c-6fa0-4655-ba28-ce2ff07d3c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:55.331898 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-ea38cf6c-6fa0-4655-ba28-ce2ff07d3c3d None None] MFA Rules not processed for user `6738a8328dfb4345b61a4356cb09b8da`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:55.333137 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-ea38cf6c-6fa0-4655-ba28-ce2ff07d3c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:55.397766 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1437/2872] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:55.428943 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0370a9-0d7f-4f93-a58b-1084d93f078a admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:55.430655 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-9a0370a9-0d7f-4f93-a58b-1084d93f078a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:55.481959 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:55.507556 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0370a9-0d7f-4f93-a58b-1084d93f078a admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:55.511727 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf2e0e-061c-4c2c-a0dd-c22a3d23c39c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:55.512360 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf2e0e-061c-4c2c-a0dd-c22a3d23c39c service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:55.512638 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf2e0e-061c-4c2c-a0dd-c22a3d23c39c service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:55.512911 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf2e0e-061c-4c2c-a0dd-c22a3d23c39c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:55.521824 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf2e0e-061c-4c2c-a0dd-c22a3d23c39c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:55.523519 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf2e0e-061c-4c2c-a0dd-c22a3d23c39c service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:55.532225 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1436/2873] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:57:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:57:55.547390 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38cf6c-6fa0-4655-ba28-ce2ff07d3c3d None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:55.611856 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38cf6c-6fa0-4655-ba28-ce2ff07d3c3d None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:55.617059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a80853-b760-48dc-b1a3-2a52447ee3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:55.617918 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a80853-b760-48dc-b1a3-2a52447ee3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:55.618197 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a80853-b760-48dc-b1a3-2a52447ee3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:55.618501 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a80853-b760-48dc-b1a3-2a52447ee3c5 admin admin] PATH_INFO: `/v3/users/6738a8328dfb4345b61a4356cb09b8da` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:55.619690 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a80853-b760-48dc-b1a3-2a52447ee3c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6738a8328dfb4345b61a4356cb09b8da)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:55.621726 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a80853-b760-48dc-b1a3-2a52447ee3c5 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:55.797586 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-69a80853-b760-48dc-b1a3-2a52447ee3c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6738a8328dfb4345b61a4356cb09b8da'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:55.804809 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-69a80853-b760-48dc-b1a3-2a52447ee3c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6738a8328dfb4345b61a4356cb09b8da'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:55.805526 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-69a80853-b760-48dc-b1a3-2a52447ee3c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6738a8328dfb4345b61a4356cb09b8da'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:55.818262 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-69a80853-b760-48dc-b1a3-2a52447ee3c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6738a8328dfb4345b61a4356cb09b8da'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:55.891793 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-69a80853-b760-48dc-b1a3-2a52447ee3c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6738a8328dfb4345b61a4356cb09b8da'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:55.898516 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1438/2874] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:57:55 2022] DELETE /identity/v3/users/6738a8328dfb4345b61a4356cb09b8da => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:55.903251 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfd039-68ec-4f67-9337-53f61f6bbb89 None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:55.903586 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfd039-68ec-4f67-9337-53f61f6bbb89 None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:55.903862 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfd039-68ec-4f67-9337-53f61f6bbb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:55.921602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-ddbfd039-68ec-4f67-9337-53f61f6bbb89 None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:55.922826 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ddbfd039-68ec-4f67-9337-53f61f6bbb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:55.980220 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1437/2875] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:55.989682 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a80853-b760-48dc-b1a3-2a52447ee3c5 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:55.991119 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-69a80853-b760-48dc-b1a3-2a52447ee3c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:55.997525 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:56.049143 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a80853-b760-48dc-b1a3-2a52447ee3c5 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:56.054104 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1dba9-2e30-4ff2-a062-2915dfa6eb58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:56.054775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1dba9-2e30-4ff2-a062-2915dfa6eb58 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:56.055059 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1dba9-2e30-4ff2-a062-2915dfa6eb58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:56.055328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1dba9-2e30-4ff2-a062-2915dfa6eb58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:56.064227 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa1dba9-2e30-4ff2-a062-2915dfa6eb58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:56.066094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa1dba9-2e30-4ff2-a062-2915dfa6eb58 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:56.075416 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1439/2876] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:57:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:57:56.488807 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfd039-68ec-4f67-9337-53f61f6bbb89 None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:56.490251 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ddbfd039-68ec-4f67-9337-53f61f6bbb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:56.502603 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfd039-68ec-4f67-9337-53f61f6bbb89 None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:56.508441 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce99a89-ebf2-4331-a5c6-9f5f17911098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:56.514017 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce99a89-ebf2-4331-a5c6-9f5f17911098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:56.514017 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce99a89-ebf2-4331-a5c6-9f5f17911098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:56.514017 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce99a89-ebf2-4331-a5c6-9f5f17911098 admin admin] PATH_INFO: `/v3/projects/0c289a6f8ab1459ead2fb60cd28d103c` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:56.514017 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce99a89-ebf2-4331-a5c6-9f5f17911098 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c289a6f8ab1459ead2fb60cd28d103c)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:56.514017 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce99a89-ebf2-4331-a5c6-9f5f17911098 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:56.634930 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-2ce99a89-ebf2-4331-a5c6-9f5f17911098 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c289a6f8ab1459ead2fb60cd28d103c', 'request_id': 'req-2ce99a89-ebf2-4331-a5c6-9f5f17911098'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:56.697583 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-2ce99a89-ebf2-4331-a5c6-9f5f17911098 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c289a6f8ab1459ead2fb60cd28d103c', 'request_id': 'req-2ce99a89-ebf2-4331-a5c6-9f5f17911098'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:56.704191 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-2ce99a89-ebf2-4331-a5c6-9f5f17911098 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c289a6f8ab1459ead2fb60cd28d103c', 'request_id': 'req-2ce99a89-ebf2-4331-a5c6-9f5f17911098'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:56.714675 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-2ce99a89-ebf2-4331-a5c6-9f5f17911098 admin admin] The token cache is being invalidate because project 0c289a6f8ab1459ead2fb60cd28d103c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:57:56.714965 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-2ce99a89-ebf2-4331-a5c6-9f5f17911098 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:56.715593 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-2ce99a89-ebf2-4331-a5c6-9f5f17911098 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:57:56.717129 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1438/2877] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:57:56 2022] DELETE /identity/v3/projects/0c289a6f8ab1459ead2fb60cd28d103c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:56.723039 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65599c-4799-47b6-957c-4da4c56d417c None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:56.723559 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65599c-4799-47b6-957c-4da4c56d417c None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:56.723665 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65599c-4799-47b6-957c-4da4c56d417c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:56.741960 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-3b65599c-4799-47b6-957c-4da4c56d417c None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:56.743220 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-3b65599c-4799-47b6-957c-4da4c56d417c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:56.803356 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1440/2878] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:56.811129 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72314858-c110-4368-9634-44e7a47b6f4d None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:56.811422 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72314858-c110-4368-9634-44e7a47b6f4d None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:56.811702 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72314858-c110-4368-9634-44e7a47b6f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:56.829784 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-72314858-c110-4368-9634-44e7a47b6f4d None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:56.831023 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-72314858-c110-4368-9634-44e7a47b6f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:56.839034 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1439/2879] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:56.843175 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b65599c-4799-47b6-957c-4da4c56d417c None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:56.853243 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b65599c-4799-47b6-957c-4da4c56d417c None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:56.858092 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ada14-3c6c-4262-af3d-43fd18a50af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:56.858775 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ada14-3c6c-4262-af3d-43fd18a50af6 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:56.859057 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ada14-3c6c-4262-af3d-43fd18a50af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:56.859328 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ada14-3c6c-4262-af3d-43fd18a50af6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:56.860042 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967ada14-3c6c-4262-af3d-43fd18a50af6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80569) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 18:57:56.860342 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967ada14-3c6c-4262-af3d-43fd18a50af6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:56.861935 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967ada14-3c6c-4262-af3d-43fd18a50af6 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:56.874463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1441/2880] 10.222.0.15 () {58 vars in 1152 bytes} [Mon Jun 27 18:57:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 18:57:56.879120 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72314858-c110-4368-9634-44e7a47b6f4d None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:56.888306 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72314858-c110-4368-9634-44e7a47b6f4d None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:56.892833 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c200b-fbb5-497e-86a5-8eb7d874b9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:56.893503 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c200b-fbb5-497e-86a5-8eb7d874b9bf admin admin] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:56.894469 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c200b-fbb5-497e-86a5-8eb7d874b9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:56.894763 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c200b-fbb5-497e-86a5-8eb7d874b9bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:56.895451 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c200b-fbb5-497e-86a5-8eb7d874b9bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:56.897344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c200b-fbb5-497e-86a5-8eb7d874b9bf admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:56.983175 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1440/2881] 10.222.0.15 () {60 vars in 1152 bytes} [Mon Jun 27 18:57:56 2022] POST /identity/v3/projects => generated 391 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:57:56.987332 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ada14-3c6c-4262-af3d-43fd18a50af6 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:56.997457 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ada14-3c6c-4262-af3d-43fd18a50af6 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:57.053334 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff362a86-2bd3-4a98-8e42-7cc7635fc78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:57.053987 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff362a86-2bd3-4a98-8e42-7cc7635fc78d admin admin] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:57.054266 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff362a86-2bd3-4a98-8e42-7cc7635fc78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:57.054568 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff362a86-2bd3-4a98-8e42-7cc7635fc78d admin admin] PATH_INFO: `/v3/users` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:57.055316 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff362a86-2bd3-4a98-8e42-7cc7635fc78d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:57.057284 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff362a86-2bd3-4a98-8e42-7cc7635fc78d admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:57.125169 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1442/2882] 10.222.0.15 () {60 vars in 1143 bytes} [Mon Jun 27 18:57:56 2022] POST /identity/v3/users => generated 296 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 18:57:57.129601 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c200b-fbb5-497e-86a5-8eb7d874b9bf admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:57.139845 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c200b-fbb5-497e-86a5-8eb7d874b9bf admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:57.145038 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfffbf7e-edce-43e0-b16a-475270f10082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:57.145674 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfffbf7e-edce-43e0-b16a-475270f10082 admin admin] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:57.145954 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfffbf7e-edce-43e0-b16a-475270f10082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:57.146309 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfffbf7e-edce-43e0-b16a-475270f10082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:57.147065 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfffbf7e-edce-43e0-b16a-475270f10082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:57.148649 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfffbf7e-edce-43e0-b16a-475270f10082 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:57.158939 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1441/2883] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:57:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 18:57:57.163330 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff362a86-2bd3-4a98-8e42-7cc7635fc78d admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:57.173161 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff362a86-2bd3-4a98-8e42-7cc7635fc78d admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:57.178286 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52db62-d366-4280-869d-8eab66057e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:57.178963 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52db62-d366-4280-869d-8eab66057e45 admin admin] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:57.179241 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52db62-d366-4280-869d-8eab66057e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:57.179514 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52db62-d366-4280-869d-8eab66057e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:57.180228 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52db62-d366-4280-869d-8eab66057e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:57.181800 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52db62-d366-4280-869d-8eab66057e45 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:57.192094 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1443/2884] 10.222.0.15 () {58 vars in 1121 bytes} [Mon Jun 27 18:57:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 18:57:57.196059 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfffbf7e-edce-43e0-b16a-475270f10082 admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:57.205782 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfffbf7e-edce-43e0-b16a-475270f10082 admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:57.210900 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14daae1-df6b-40c9-81d5-e7377da5b56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:57.211757 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14daae1-df6b-40c9-81d5-e7377da5b56d admin admin] REQUEST_METHOD: `PUT` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:57.212038 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14daae1-df6b-40c9-81d5-e7377da5b56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:57.212314 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14daae1-df6b-40c9-81d5-e7377da5b56d admin admin] PATH_INFO: `/v3/projects/80cc1228974f4febadb2ab6257d6d4e1/users/f38fc86791b84944a25b421787f059a9/roles/b45aa1c36d974fe79c809afbfcd01945` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:57.224270 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:57.227900 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14daae1-df6b-40c9-81d5-e7377da5b56d admin admin] RBAC: Authorizing `identity:create_grant(project_id=80cc1228974f4febadb2ab6257d6d4e1, user_id=f38fc86791b84944a25b421787f059a9, role_id=b45aa1c36d974fe79c809afbfcd01945)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:57.230624 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14daae1-df6b-40c9-81d5-e7377da5b56d admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:57.291664 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1442/2885] 10.222.0.15 () {60 vars in 1482 bytes} [Mon Jun 27 18:57:57 2022] PUT /identity/v3/projects/80cc1228974f4febadb2ab6257d6d4e1/users/f38fc86791b84944a25b421787f059a9/roles/b45aa1c36d974fe79c809afbfcd01945 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:57:57.301814 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17e3c5-ebff-451c-8da0-0300957b96bf None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:57.302102 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17e3c5-ebff-451c-8da0-0300957b96bf None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:57.302376 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17e3c5-ebff-451c-8da0-0300957b96bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:57.326399 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:57:57.329219 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-bb17e3c5-ebff-451c-8da0-0300957b96bf None None] MFA Rules not processed for user `f38fc86791b84944a25b421787f059a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:57:57.330457 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-bb17e3c5-ebff-451c-8da0-0300957b96bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:57.395050 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1444/2886] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:57:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:57:57.405712 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14daae1-df6b-40c9-81d5-e7377da5b56d admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:57:57.407463 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-e14daae1-df6b-40c9-81d5-e7377da5b56d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:57:57.464915 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14daae1-df6b-40c9-81d5-e7377da5b56d admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:57:57.468536 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc80ec-2d45-4568-a260-035ccd92b4b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4f6f8e0a0cb4c9a9dd0d99fc0022235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:57:57.469174 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc80ec-2d45-4568-a260-035ccd92b4b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:57:57.469457 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc80ec-2d45-4568-a260-035ccd92b4b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:57:57.469730 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc80ec-2d45-4568-a260-035ccd92b4b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:57:57.478640 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc80ec-2d45-4568-a260-035ccd92b4b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:57:57.480064 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc80ec-2d45-4568-a260-035ccd92b4b6 service cinder] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:57:57.489370 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1443/2887] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:57:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:58:07.399337 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=26 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:58:07.484745 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:58:15.997221 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491de5a-6a9f-4c28-808b-06d539daa5ab None None] REQUEST_METHOD: `POST` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:58:15.997738 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491de5a-6a9f-4c28-808b-06d539daa5ab None None] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:58:15.997848 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491de5a-6a9f-4c28-808b-06d539daa5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:58:16.015784 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.auth.core [None req-f491de5a-6a9f-4c28-808b-06d539daa5ab None None] MFA Rules not processed for user `b10a90cce1b048d7bd55938811301ab4`. Rule list: `[]` (Enabled: `True`). {{(pid=80569) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:58:16.017046 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-f491de5a-6a9f-4c28-808b-06d539daa5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:58:16.071779 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1445/2888] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:58:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:58:18.213718 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc80ec-2d45-4568-a260-035ccd92b4b6 service cinder] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:58:18.215248 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-c8bc80ec-2d45-4568-a260-035ccd92b4b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:58:18.276602 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc80ec-2d45-4568-a260-035ccd92b4b6 service cinder] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:58:18.280585 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b2b0c-35e8-4799-8644-d9e22d739d1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:58:18.281285 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b2b0c-35e8-4799-8644-d9e22d739d1e service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:58:18.281573 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b2b0c-35e8-4799-8644-d9e22d739d1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:58:18.281878 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b2b0c-35e8-4799-8644-d9e22d739d1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:58:18.284108 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-131b2b0c-35e8-4799-8644-d9e22d739d1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:58:18.296456 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b2b0c-35e8-4799-8644-d9e22d739d1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:58:18.297925 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b2b0c-35e8-4799-8644-d9e22d739d1e service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:58:18.308253 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1444/2889] 10.222.0.15 () {60 vars in 1396 bytes} [Mon Jun 27 18:58:18 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3244 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 18:58:19.179683 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491de5a-6a9f-4c28-808b-06d539daa5ab None None] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:58:19.246175 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491de5a-6a9f-4c28-808b-06d539daa5ab None None] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:58:19.250706 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f5b35-08d5-4e00-9461-9ec026b38d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:58:19.251850 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f5b35-08d5-4e00-9461-9ec026b38d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:58:19.252125 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f5b35-08d5-4e00-9461-9ec026b38d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:58:19.252396 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f5b35-08d5-4e00-9461-9ec026b38d7a admin admin] PATH_INFO: `/v3/users/f38fc86791b84944a25b421787f059a9` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:58:19.253610 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f5b35-08d5-4e00-9461-9ec026b38d7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f38fc86791b84944a25b421787f059a9)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:58:19.255677 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f5b35-08d5-4e00-9461-9ec026b38d7a admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:58:19.498073 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-727f5b35-08d5-4e00-9461-9ec026b38d7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f38fc86791b84944a25b421787f059a9'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:58:19.506162 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-727f5b35-08d5-4e00-9461-9ec026b38d7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f38fc86791b84944a25b421787f059a9'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:58:19.506950 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-727f5b35-08d5-4e00-9461-9ec026b38d7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f38fc86791b84944a25b421787f059a9'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:58:19.519424 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-727f5b35-08d5-4e00-9461-9ec026b38d7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f38fc86791b84944a25b421787f059a9'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:58:19.594937 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-727f5b35-08d5-4e00-9461-9ec026b38d7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f38fc86791b84944a25b421787f059a9'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:58:19.603507 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1446/2890] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 18:58:19 2022] DELETE /identity/v3/users/f38fc86791b84944a25b421787f059a9 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:58:19.608155 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d0a5f-b446-4c5c-9d05-990532fec43a None None] REQUEST_METHOD: `POST` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:58:19.608999 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d0a5f-b446-4c5c-9d05-990532fec43a None None] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:58:19.609287 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d0a5f-b446-4c5c-9d05-990532fec43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:58:19.628021 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.auth.core [None req-fd6d0a5f-b446-4c5c-9d05-990532fec43a None None] MFA Rules not processed for user `3d0ddf436e7443679ab397306fca9e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 18:58:19.629180 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-fd6d0a5f-b446-4c5c-9d05-990532fec43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:58:19.690334 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1445/2891] 10.222.0.15 () {58 vars in 957 bytes} [Mon Jun 27 18:58:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 18:58:19.709341 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f5b35-08d5-4e00-9461-9ec026b38d7a admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:58:19.709341 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-727f5b35-08d5-4e00-9461-9ec026b38d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:58:19.733254 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:58:19.772553 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f5b35-08d5-4e00-9461-9ec026b38d7a admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:58:19.780231 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eff426-a8c2-42fd-b337-a4fea1d35466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:58:19.780898 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eff426-a8c2-42fd-b337-a4fea1d35466 service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:58:19.781181 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eff426-a8c2-42fd-b337-a4fea1d35466 service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:58:19.781463 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eff426-a8c2-42fd-b337-a4fea1d35466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:58:19.792812 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eff426-a8c2-42fd-b337-a4fea1d35466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:58:19.794255 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eff426-a8c2-42fd-b337-a4fea1d35466 service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:58:19.804628 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1447/2892] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:58:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:58:20.392460 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d0a5f-b446-4c5c-9d05-990532fec43a None None] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:58:20.394011 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-fd6d0a5f-b446-4c5c-9d05-990532fec43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:58:20.412203 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d0a5f-b446-4c5c-9d05-990532fec43a None None] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:58:20.427697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a7f1f-d543-430c-acf0-22a2dac2a62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:58:20.428574 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a7f1f-d543-430c-acf0-22a2dac2a62a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:58:20.428838 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a7f1f-d543-430c-acf0-22a2dac2a62a admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:58:20.429114 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a7f1f-d543-430c-acf0-22a2dac2a62a admin admin] PATH_INFO: `/v3/projects/80cc1228974f4febadb2ab6257d6d4e1` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:58:20.431659 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a7f1f-d543-430c-acf0-22a2dac2a62a admin admin] RBAC: Authorizing `identity:delete_project(project_id=80cc1228974f4febadb2ab6257d6d4e1)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:58:20.433615 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a7f1f-d543-430c-acf0-22a2dac2a62a admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:58:20.557180 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-eb8a7f1f-d543-430c-acf0-22a2dac2a62a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80cc1228974f4febadb2ab6257d6d4e1', 'request_id': 'req-eb8a7f1f-d543-430c-acf0-22a2dac2a62a'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:58:20.623242 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-eb8a7f1f-d543-430c-acf0-22a2dac2a62a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80cc1228974f4febadb2ab6257d6d4e1', 'request_id': 'req-eb8a7f1f-d543-430c-acf0-22a2dac2a62a'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:58:20.632236 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-eb8a7f1f-d543-430c-acf0-22a2dac2a62a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80cc1228974f4febadb2ab6257d6d4e1', 'request_id': 'req-eb8a7f1f-d543-430c-acf0-22a2dac2a62a'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:58:20.646984 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-eb8a7f1f-d543-430c-acf0-22a2dac2a62a admin admin] The token cache is being invalidate because project 80cc1228974f4febadb2ab6257d6d4e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 18:58:20.647432 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-eb8a7f1f-d543-430c-acf0-22a2dac2a62a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:58:20.648832 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-eb8a7f1f-d543-430c-acf0-22a2dac2a62a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 18:58:20.657344 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1446/2893] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 18:58:20 2022] DELETE /identity/v3/projects/80cc1228974f4febadb2ab6257d6d4e1 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 18:58:27.967713 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eff426-a8c2-42fd-b337-a4fea1d35466 service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 18:58:27.969160 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-a0eff426-a8c2-42fd-b337-a4fea1d35466 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:58:28.059187 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eff426-a8c2-42fd-b337-a4fea1d35466 service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 18:58:28.119211 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b56a96-1cb7-47a8-b43e-42f608f741e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cd6f0a1286243e4add692db01028205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 18:58:28.119877 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b56a96-1cb7-47a8-b43e-42f608f741e0 service placement] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 18:58:28.120157 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b56a96-1cb7-47a8-b43e-42f608f741e0 service placement] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 18:58:28.120430 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b56a96-1cb7-47a8-b43e-42f608f741e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 18:58:28.122172 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-92b56a96-1cb7-47a8-b43e-42f608f741e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 18:58:28.134134 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b56a96-1cb7-47a8-b43e-42f608f741e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 18:58:28.135738 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b56a96-1cb7-47a8-b43e-42f608f741e0 service placement] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 18:58:28.147060 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1448/2894] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 18:58:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 18:58:30.643453 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 18:58:38.141967 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=82 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 19:00:09.461491 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a7f1f-d543-430c-acf0-22a2dac2a62a admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 19:00:09.462933 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-eb8a7f1f-d543-430c-acf0-22a2dac2a62a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 19:00:09.526006 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a7f1f-d543-430c-acf0-22a2dac2a62a admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 19:00:09.529627 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead4f4c6-7d06-4d41-a944-ed5ec65e40ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 19:00:09.530268 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4f4c6-7d06-4d41-a944-ed5ec65e40ce service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 19:00:09.530576 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4f4c6-7d06-4d41-a944-ed5ec65e40ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 19:00:09.530852 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4f4c6-7d06-4d41-a944-ed5ec65e40ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 19:00:09.533835 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-ead4f4c6-7d06-4d41-a944-ed5ec65e40ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 19:00:09.625519 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead4f4c6-7d06-4d41-a944-ed5ec65e40ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 19:00:09.627204 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead4f4c6-7d06-4d41-a944-ed5ec65e40ce service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 19:00:09.647952 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1447/2895] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 19:00:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 19:00:10.830303 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b56a96-1cb7-47a8-b43e-42f608f741e0 service placement] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 19:00:10.839551 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b56a96-1cb7-47a8-b43e-42f608f741e0 service placement] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 19:00:10.843144 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a70d5-196d-4126-b2e8-2fdfcc7fa5fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 19:00:10.844248 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a70d5-196d-4126-b2e8-2fdfcc7fa5fd service neutron] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 19:00:10.844625 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a70d5-196d-4126-b2e8-2fdfcc7fa5fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 19:00:10.844904 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a70d5-196d-4126-b2e8-2fdfcc7fa5fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 19:00:10.846801 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-273a70d5-196d-4126-b2e8-2fdfcc7fa5fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 19:00:10.860703 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a70d5-196d-4126-b2e8-2fdfcc7fa5fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 19:00:10.862122 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a70d5-196d-4126-b2e8-2fdfcc7fa5fd service neutron] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 19:00:10.872995 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1449/2896] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 19:00:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 19:00:19.645351 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=66 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 19:00:20.868159 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=6 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 19:00:22.667549 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead4f4c6-7d06-4d41-a944-ed5ec65e40ce service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 19:00:22.677253 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead4f4c6-7d06-4d41-a944-ed5ec65e40ce service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 19:00:22.680908 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00194e92-e570-467a-807c-705ad2939ede service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbe621916dbe4f1a8686097fe2bdf762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 19:00:22.681532 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00194e92-e570-467a-807c-705ad2939ede service neutron] REQUEST_METHOD: `GET` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 19:00:22.681816 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00194e92-e570-467a-807c-705ad2939ede service neutron] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 19:00:22.682090 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00194e92-e570-467a-807c-705ad2939ede service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 19:00:22.683885 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-00194e92-e570-467a-807c-705ad2939ede service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 19:00:22.751562 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00194e92-e570-467a-807c-705ad2939ede service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 19:00:22.753317 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00194e92-e570-467a-807c-705ad2939ede service neutron] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 19:00:22.763567 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1448/2897] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 19:00:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 19:00:29.701661 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a70d5-196d-4126-b2e8-2fdfcc7fa5fd service neutron] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 19:00:29.702693 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-273a70d5-196d-4126-b2e8-2fdfcc7fa5fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 19:00:29.719727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a70d5-196d-4126-b2e8-2fdfcc7fa5fd service neutron] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 19:00:29.724224 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de500d-126f-4494-92ea-2dc7371d8610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 19:00:29.725072 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de500d-126f-4494-92ea-2dc7371d8610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 19:00:29.725350 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de500d-126f-4494-92ea-2dc7371d8610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 19:00:29.725622 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de500d-126f-4494-92ea-2dc7371d8610 admin admin] PATH_INFO: `/v3/users/b10a90cce1b048d7bd55938811301ab4` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 19:00:29.726822 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de500d-126f-4494-92ea-2dc7371d8610 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b10a90cce1b048d7bd55938811301ab4)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 19:00:29.728708 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de500d-126f-4494-92ea-2dc7371d8610 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 19:00:29.920433 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-91de500d-126f-4494-92ea-2dc7371d8610 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b10a90cce1b048d7bd55938811301ab4'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 19:00:29.928308 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-91de500d-126f-4494-92ea-2dc7371d8610 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b10a90cce1b048d7bd55938811301ab4'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 19:00:29.928971 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-91de500d-126f-4494-92ea-2dc7371d8610 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b10a90cce1b048d7bd55938811301ab4'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 19:00:29.941032 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-91de500d-126f-4494-92ea-2dc7371d8610 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b10a90cce1b048d7bd55938811301ab4'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 19:00:30.026619 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-91de500d-126f-4494-92ea-2dc7371d8610 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b10a90cce1b048d7bd55938811301ab4'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 19:00:30.034977 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1450/2898] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 19:00:29 2022] DELETE /identity/v3/users/b10a90cce1b048d7bd55938811301ab4 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 19:00:30.285803 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00194e92-e570-467a-807c-705ad2939ede service neutron] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 19:00:30.299713 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00194e92-e570-467a-807c-705ad2939ede service neutron] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 19:00:30.357264 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130018f-8e24-495e-81ba-41fdc7d0145e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 19:00:30.358110 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130018f-8e24-495e-81ba-41fdc7d0145e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 19:00:30.358408 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130018f-8e24-495e-81ba-41fdc7d0145e admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 19:00:30.358684 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130018f-8e24-495e-81ba-41fdc7d0145e admin admin] PATH_INFO: `/v3/projects/302e8c5b4c1f422fbd5cf5602a094d2e` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 19:00:30.361322 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130018f-8e24-495e-81ba-41fdc7d0145e admin admin] RBAC: Authorizing `identity:delete_project(project_id=302e8c5b4c1f422fbd5cf5602a094d2e)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 19:00:30.363298 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130018f-8e24-495e-81ba-41fdc7d0145e admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 19:00:30.466960 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7130018f-8e24-495e-81ba-41fdc7d0145e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '302e8c5b4c1f422fbd5cf5602a094d2e', 'request_id': 'req-7130018f-8e24-495e-81ba-41fdc7d0145e'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 19:00:30.540697 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7130018f-8e24-495e-81ba-41fdc7d0145e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '302e8c5b4c1f422fbd5cf5602a094d2e', 'request_id': 'req-7130018f-8e24-495e-81ba-41fdc7d0145e'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 19:00:30.547434 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7130018f-8e24-495e-81ba-41fdc7d0145e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '302e8c5b4c1f422fbd5cf5602a094d2e', 'request_id': 'req-7130018f-8e24-495e-81ba-41fdc7d0145e'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 19:00:30.557749 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7130018f-8e24-495e-81ba-41fdc7d0145e admin admin] The token cache is being invalidate because project 302e8c5b4c1f422fbd5cf5602a094d2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 19:00:30.558028 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7130018f-8e24-495e-81ba-41fdc7d0145e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 19:00:30.558667 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-7130018f-8e24-495e-81ba-41fdc7d0145e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 19:00:30.560195 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1449/2899] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 19:00:30 2022] DELETE /identity/v3/projects/302e8c5b4c1f422fbd5cf5602a094d2e => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 19:00:30.564783 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de500d-126f-4494-92ea-2dc7371d8610 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 19:00:30.566180 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-91de500d-126f-4494-92ea-2dc7371d8610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 19:00:30.580001 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de500d-126f-4494-92ea-2dc7371d8610 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 19:00:30.636545 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb96f5-4ecf-4571-b7e5-35c612592a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 19:00:30.637412 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb96f5-4ecf-4571-b7e5-35c612592a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 19:00:30.637694 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb96f5-4ecf-4571-b7e5-35c612592a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 19:00:30.637970 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb96f5-4ecf-4571-b7e5-35c612592a84 admin admin] PATH_INFO: `/v3/users/a70f695bbb7b494ab85f253ad5285e5a` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 19:00:30.639164 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb96f5-4ecf-4571-b7e5-35c612592a84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a70f695bbb7b494ab85f253ad5285e5a)` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 19:00:30.642182 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb96f5-4ecf-4571-b7e5-35c612592a84 admin admin] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 19:00:30.828727 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-b5fb96f5-4ecf-4571-b7e5-35c612592a84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a70f695bbb7b494ab85f253ad5285e5a'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 19:00:30.835611 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-b5fb96f5-4ecf-4571-b7e5-35c612592a84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a70f695bbb7b494ab85f253ad5285e5a'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 19:00:30.836281 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-b5fb96f5-4ecf-4571-b7e5-35c612592a84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a70f695bbb7b494ab85f253ad5285e5a'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 19:00:30.848213 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-b5fb96f5-4ecf-4571-b7e5-35c612592a84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a70f695bbb7b494ab85f253ad5285e5a'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 19:00:30.849854 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 19:00:30.934729 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.notifications [None req-b5fb96f5-4ecf-4571-b7e5-35c612592a84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a70f695bbb7b494ab85f253ad5285e5a'} {{(pid=80569) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 19:00:30.941732 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1451/2900] 10.222.0.15 () {58 vars in 1223 bytes} [Mon Jun 27 19:00:30 2022] DELETE /identity/v3/users/a70f695bbb7b494ab85f253ad5285e5a => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 19:00:31.190482 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130018f-8e24-495e-81ba-41fdc7d0145e admin admin] Authenticating user token {{(pid=80570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 19:00:31.192024 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.fernet_utils [None req-7130018f-8e24-495e-81ba-41fdc7d0145e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 19:00:31.205135 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130018f-8e24-495e-81ba-41fdc7d0145e admin admin] Validating token access rules against request {{(pid=80570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 19:00:31.226328 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 19:00:31.263702 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e11f4-2ad1-42ec-a626-3a2b1441d051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d0ddf436e7443679ab397306fca9e90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9ffe361a6094e199bb35c633199d02f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 19:00:31.264587 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e11f4-2ad1-42ec-a626-3a2b1441d051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 19:00:31.264875 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e11f4-2ad1-42ec-a626-3a2b1441d051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 19:00:31.265155 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e11f4-2ad1-42ec-a626-3a2b1441d051 admin admin] PATH_INFO: `/v3/projects/7f7af5ede07d48e3a9c650602a70db45` {{(pid=80570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 19:00:31.266371 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e11f4-2ad1-42ec-a626-3a2b1441d051 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f7af5ede07d48e3a9c650602a70db45)` {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 19:00:31.271428 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e11f4-2ad1-42ec-a626-3a2b1441d051 admin admin] RBAC: Authorization granted {{(pid=80570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 19:00:31.387686 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-117e11f4-2ad1-42ec-a626-3a2b1441d051 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f7af5ede07d48e3a9c650602a70db45', 'request_id': 'req-117e11f4-2ad1-42ec-a626-3a2b1441d051'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 19:00:31.454985 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-117e11f4-2ad1-42ec-a626-3a2b1441d051 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f7af5ede07d48e3a9c650602a70db45', 'request_id': 'req-117e11f4-2ad1-42ec-a626-3a2b1441d051'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 19:00:31.461839 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-117e11f4-2ad1-42ec-a626-3a2b1441d051 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f7af5ede07d48e3a9c650602a70db45', 'request_id': 'req-117e11f4-2ad1-42ec-a626-3a2b1441d051'} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 19:00:31.472976 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-117e11f4-2ad1-42ec-a626-3a2b1441d051 admin admin] The token cache is being invalidate because project 7f7af5ede07d48e3a9c650602a70db45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 19:00:31.473264 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-117e11f4-2ad1-42ec-a626-3a2b1441d051 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 19:00:31.473898 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG keystone.notifications [None req-117e11f4-2ad1-42ec-a626-3a2b1441d051 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 19:00:31.475479 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: [pid: 80570|app: 0|req: 1450/2901] 10.222.0.15 () {58 vars in 1232 bytes} [Mon Jun 27 19:00:31 2022] DELETE /identity/v3/projects/7f7af5ede07d48e3a9c650602a70db45 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 19:00:40.986745 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 19:00:41.469951 dv-cinder-844601-6-iscsi devstack@keystone.service[80570]: DEBUG dbcounter [-] [80570] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 19:03:32.969545 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb96f5-4ecf-4571-b7e5-35c612592a84 admin admin] Authenticating user token {{(pid=80569) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 19:03:32.970986 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-b5fb96f5-4ecf-4571-b7e5-35c612592a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 19:03:33.041962 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb96f5-4ecf-4571-b7e5-35c612592a84 admin admin] Validating token access rules against request {{(pid=80569) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 19:03:33.102238 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e1738-182e-491c-b2c8-3a4d631a788a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cd6f0a1286243e4add692db01028205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '84ecf552f8364884b4aeff3428cb81e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80569) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 19:03:33.102915 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e1738-182e-491c-b2c8-3a4d631a788a service placement] REQUEST_METHOD: `GET` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 19:03:33.103215 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e1738-182e-491c-b2c8-3a4d631a788a service placement] SCRIPT_NAME: `/identity` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 19:03:33.103492 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e1738-182e-491c-b2c8-3a4d631a788a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80569) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 19:03:33.105256 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.fernet_utils [None req-0c9e1738-182e-491c-b2c8-3a4d631a788a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80569) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 19:03:33.117528 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e1738-182e-491c-b2c8-3a4d631a788a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 19:03:33.119175 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e1738-182e-491c-b2c8-3a4d631a788a service placement] RBAC: Authorization granted {{(pid=80569) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 19:03:33.129503 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: [pid: 80569|app: 0|req: 1452/2902] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Jun 27 19:03:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 19:03:43.124596 dv-cinder-844601-6-iscsi devstack@keystone.service[80569]: DEBUG dbcounter [-] [80569] Writing DB stats keystone:SELECT=58 {{(pid=80569) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}